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FOREWORD 


The  25th  Conference  on  the  Design  of  Experiments  in  Army  Research, 
Development  and  Testing  (DOE)  was  held  1 7” 1 9  October  1979  at  the 
U.  S.  Army  Natick  Research  and  Development  Command  in  Natick, 
Massachusetts.  This  was  the  second  time  in  the  history  of  these 
conferences  that  this  Army  base  has  provided  the  facilities  to  conduct 
one  of  these  scientific  meetings.  The  fourth  conference  in  the  series 

was  held  here.  At  that  time,  the  base  was  called  the  Quartermaster 

■> 

Research  and  Engineering  Center. 

The  original  format  for  the  DOE  Conferences,  which  are  under  the 
auspices  of  the  Army  Mathematics  Steering  Committee  (AMSC) ,  was  outlined 
by  the  eminent  statistician,  Professor  Samuel  S.  Wilks,  who  served  as 
conference  chairman  until  his  death,  lhrough  these  symposia  the 
AMSC  hopes  to  Introduce  and  encourage  the  use  of  the  latest  statistical 
and  design  techniques  Into  the  research,  development  and  testing 
conducted  by  the  Army's  scientific  and  engineering  personnel.  It  is 
believed  that  this  purpose  can  be  best  pursued  by  holding  these  meetings 
at  various  government  installations  throughout  the  country. 

Several  features  in  this  year's  agenda  pointed  out  the  special 
significance  of  this,  the  Silver  Anniversary  of  these  meetings.  The 
program  was  dedicated  to  Dr.  Francis  G.  Dressel,  formerly  Professor 
of  Mathematics  at  Duke  University.  For  the  past  25  years  Dr.  Dressel 
has  coordinated  the  conference  programming  and  local  arrangements, 


and  has  edited  proceedings  of  this  and  other  AMSC  sponsored  conferences. 
The  00E  Program  Committee  Invited  Dr.  Dressel's  friends  to  Join  In 
expressing  their  appreciation  for  this  loyal  service. 


The  Program  was  to  begin  with  an  address  by  Dr.  Frank  E.  Grubbs, 
formerly  of  the  U.  S.  Army  Ballistic  Research  Laboratory,  and  a 
renowned  statistician.  His  talk  was  entitled,  "A  Quarter  Century 
of  Army  Design  of  Experiments  Conferences11.  This  was  to  be  followed 
by  an  address  entitled,  "Summarizing  the  Results  of  a  Series  of 
Experiments1.1  to  be  given  by  Dr.  William  G.  Cochran,  Professor  of 
Statistics  Emeritus,  Harvard  University.  Unfortunately,  both  of 
these  gentlemen  were  unable  to  attend  the  conference.  Their 
addresses  were  read,  respectively,  by  Dr.  Dressel  and  Dr.  Herman 
Chernoff,  Professor  of  Mathematics  at  the  Massachusetts  Institute 
of  Technology.  (Dr.  Grubbs  was  also  unable  to  serve  as  Master  of 
Ceremonies  at  the  banquet  and  to  make  the  presentation  of  the  Samuel 
S.  Wilks  Memorial  Medal.  These  duties  were  taken  over  by  Dr.  Robert 
Launer  of  the  Army  Research  Office.)  The  other  Invited  speakers 
and  trie  I  r  topics  are  noted  below. 


Speaker  and  Affiliation 


Tl tie  of  Address 


Mr.  A1  L.  May 
Plllsbury  Research  Labs 


DESIGNED  EXPERIMENTS  III  SENSORY 
TESTING 


Dr.  Ray  E.  Schafer 
Hughes  Aircraft  Company 


COMPUTER  AIDED  HYPOTHESIS  TESTS  - 
THE  BIRNBAUM  TEST 


Professor  Warren  Stewart  NEW  ALGORITHMS  FOR  NONLINEAR  LEAST 

Mathematics  Research  Center  SQUARES  AND  BAYESIAN  PARAMETER 

and  University  of  Wisconsin  ESTIMATION 

Professor  Marvin  Zelen  ETHICS  AND  STRATEGY  IN  THERAPEUTIC 

Harvard  School  of  Public  INVESTIGATIONS 

Sciences 


The  members  of  the  AMSC  take  this  opportunity  to  express  their  thanks 
to  the  speakers  and  other  research  workers  who  partlcipatod  In  the 
meeting;  to  Colonel  H.  F.  Penny,  Commanding  Officer  of  the  U.  S.  Army 
Natick  Research  and  Development  Command,  for  making  available  the 
excellent  facilities  of  his  organization  for  the  conference;  and 
to  Mr.  Donald  Kass  who  so  ably  handled  the  details  of  the  local 
arrangements  for  this  meeting.  The  AMSC  Is  making  available  most  of 
the  papers  presented  at  this  meeting  In  the  present  form  In  order 
to  encourage  wider  use  of  modern  statistical  principles  of  the  design 
of  experiments  In  research,  development  and  testing  work  of  concern 
to  the  Army. 
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THE  TWENTY-FIFTH  ANNIVERSARY  OF 
THE  CONFERENCES  ON 
THE  DESIGN  OF  EXPERIMENTS  IN 
ARMY  RESEARCH,  DEVELOPMENT  AND  TESTING 


THIS  PAMPHLET  CONTAINS  THE  NAMES  OF  INVITED  SPEAKERS 
SELECTED  FROM  THE  PROGRAMS  OF 

THE  FIRST  TWENTY-FIVE  MEETINGS  IN  THIS  SERIES 

FOLLOWED  BY  A  L?<T  OF 
RECIPIENTS  OF  THE  WILKS  MEMORIAL  AWARD 


These  Conferences  Are  Sponsored  By 
The  Army  Mathematics  Steering  Conwlttee 

U.  S.  Army  Reseerch  Office 
Research  Triangle  Park,  North  Carolina 


CONFERENCES  ON  THE  DESIGN  OF  EXPERIMENTS  IN 
ARMY  RESEARCH,  DEVELOPMENT  AND  TESTING 

List  of  Invited  Speakers  at 
the  First  Twenty-Five  Meetings 

First  Conference:  19-21  Octobar  1955,  Diamond  Ordnanee  Fuza 
Laboratory  and  National  Buraau  of  Standards 

Professor  W.  G.  Cochran 

THE  PHILOSOPHY  UNDERLYING  THE  DESIGN  OF  EXPERIMENTS 
Or.  Churchill  Elsenhart 

THE  PRINCIPLE  OF  RANDOMIZATION  IN  THE  DESIGN  OF  EXPERIMENTS 
Dr.  M.  E.  Tarry 

FINDING  OPTIMUM  CONDITIONS  BV  EXPERIMENTATION 
Profassor  John  Tukay  (Chairman) 

PANEL  DISCUSSION  ON  HOW  AND  WHERE  DO  STATISTICIANS  FIT 
IN.  (THE  OTHERS  ON  THIS  PANEL  WEREi  MR.  CUTHBERT  DANIEL, 

MS.  8ESSE  DAV,  DR.  CHURCHILL  EISENHART,  DR.  M.  E.  TERRV, 

AND  PROFESSOR  S.  S.  WILKS). 

Dr.  W.  J.  Youden 

DESIGN  OF  EXPERIMENTS  IN  INDUSTRIAL  RESEARCH  AND  DEVELOPMENT 

Second  Confaranca:  17-19  Octobar  1956,  Diamond  Ordnance  Fuza 

Laboratory  and  the  National  Buraau  of  Standards 

Or.  C.  A.  Bennett 

THE  PREDESIGN  PHASE  OF  URGE  SAMPLE  EXPERIMENTS 
Professor  R.  A.  Bradley 

RECENT  kESEARCH  IN  STATISTICAL  PROBLEMS  IN  SUBJECTIVE  TESTING 
Professor  B.  G.  Greenberg 

APPLICATION  OF  ORDER  STATISTICS  IN  MEDICAL  EXPERIMENTS 
Professor  G.  E.  Nicholson,  Jr. 

THE  PUNNING  OF  EXPERIMENTS  IN  THE  PRESENCE  OF  VARIATION 
Dr.  M.  B.  Wllk 

DERIVED  LINEAR  MODELS  IN  THE  ANALYSIS  OF  VARIANCE 


X 


Dr.  Jerome  Cornfield 

CHOICE  OF  ERROR  IN  THE  DESIGN  OF  EXPERIMENTS 

Third  Conference:  16-18  October  1957,  Diamond  Ordnence  Fuze  Laboratory 
and  the  National  Bureau  of  Standards 

Professor  Benjamin  Epstein 
LIFE  TESTING 

Sir  K.  A.  Fisher 

PRACTICAL  PROBLEMS  IN  EXPERIMENTAL  DESIGN 
Professor  H.  0.  Hartley 

CHANGES  IN  THE  OUTLOOK  OF  STATISTICS  BROUGHT  ABOUT  BY  MODERN 
COMPUTERS 

Dr.  A.  W.  Marshall 

EXPERIMENTATION  BY  SIMULATION  AND  MONTE  CARLO 

Fourth  Conference:  22-24  October  196B,  The  Quartermaster  Research 
and  Engineering  Center,  Natick 

Mr.  C.  1,  Bliss 

SOME  STATISTICAL  ASPECTS  OF  PREFERENCE  STUDIES 
Professor  A.  C,  Cohen 

SIMPLIFIED  COMPUTATIONAL  PROCEDURES  FOR  ESTIMATING  PARAMETERS 
OF  A  NORMAL  DISTRIBUTION  FROM  RESTRICTED  SAMPLES 

Dr.  A.  W.  Kimball 

ERRORS  OF  THE  THIRD  KIND  IN  STATISTICAL  CONSULTING 
Professor  C.  F,  Kossack 

THE  AASHC  ROAD  TEST  AS  AN  EXAMPLE  OF  URGE  SCALE  TESTS 
Mr.  L.  H.  C.  Tippett 

STATISTICAL  UETHOVS  APPLIED  TO  THE  TEXTILE  INDUSTRY 


XI 


Fifth  Conference;  4-6  November  1969,  The  U.  S.  Army  Biological 
Warfare  Laboratories,  Ft.  Detrlck 

Dr.  Joseph  Berkson 
THE  MEASURE  OF  DEATH 

Dr.  H.  A.  David 

THE  METHOD  OF  PAIRED  COMPARISONS 
Or.  0.  B.  DeLury 

SAMPLING  IN  BIOLOGICAL  POPULATIONS 

Dr.  W.  J.  Dixon 
MEDICAL  HEALTH  STATISTICS 

Dr.  N.  E.  Golovin 

PREDICTION  OF  THE  RELIABILITY  OF  COMPLEX  SYSTEMS 
Dr.  Richard  Weiss 

THE  ARMY  RESEARCH  AND  DEYFLOPMENT  PROGRAM  AS  IT  RELATES  TO 
THE  CIVIL  ECONOMY 

Sixth  Conference:  19-21  October  1960,  The  Ballistic  Research 
Laboratory 

Dr.  James  R.  Ouffett 
RELIABILITY 

Professor  F.  J.  Anscombe 
EXAMINATION  OF  RESIDUALS 

Dr.  W.  S.  Connor 

DEVELOPMENTS  IN  THE  DESIGN  OF  EXPERIMENTS 
Dr.  0.  E.  Jackson 

MULTIVARIATE  ANALYSIS  ILLUSTRATED  BY  NIXE-HERCULESi 
I.  SEPARATION  OF  PRODUCT  AND  MEASUREMENT  VARIABILITY 
II.  ACCEPTANCE  SAMPLING 

Professor  G.  E.  P.  Box  (Chairman) 

PANEL  DISCUSSION  ON  COMMON  PITFALLS  IN  THE  DESIGN  OF 
EXPERIMENTS.  (OTHERS  ON  THE  PANEL  WERE  MR.  CUTH8ERT  DANIEL, 
OR.  J.  S.  HUNTER,  PR.  W,  J.  YOUPEN  AND  PR.  MARVIN  IELEN). 
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Seventh  Conference  18-20  October  1961,  U.  S.  Signal  Research 
and  Development  Laboratory,  Ft-  Monmouth 

Or.  G.  A.  Watterson 

TIME  SERIES  AMD  SPECTRAL  ANALYSIS 

Dr.  0.  M.  Hammerslay 
MUTE  CAR  LA  METHODS 

Dr.  R.  L.  Anderson 

DESIGNS  FOR  ESTIMATING  VARIANCE  COMPONENTS 

Dr,  G.  S.  Watson 
HAZARD  ANALYSIS 

Professor  Robert  M.  Thrall  (Chairman) 

PANEL  DISCUSSION  ON  SIMULATION.  (OTHERS  ON  THE  PANEL  WERE 
COL.  A.  W.  DEQOOV,  DR.  JOHN  HAMMERSLEY ,  MR.  JOHN  H.  MOSS  AND 
PR.  GUSTAVE  ROBSON). 

Eighth  Conference!  24-26  October  1962,  Walter  Reed  Army  Institute 
of  Research 

Professor  Egon  S.  Pearson 

A  STATISTICIAN'S  PLACE  IN  ASSESSING  THE  LIKELY  OPERATIONAL 
PERFORMANCE  OF  ARMY  WEAPONS  AND  EQUIPMENT 

Or.  Marvin  A.  Schnelderman 
A  GENERAL  SURVEY  OF  SCREENING  THEORY 

Professor  Hitman  Chernoff 
OPTIMAL  DESIGN  EXPERIMENTS 

Dr.  R.  P.  Abel  son 

AN  EXPERIMENTAL  DESIGN  FOR  DECISIONS  UNDER  UNCERTAINTY 

Dr.  H.  C.  Batson 
8I0-ASSAV 

Dr.  H»ro1d  F.  Dorn  (Chairman) 

PANEL  DISCUSSION  ON  DIET  AND  HEART  DISEASE.  {OTHERS  ON 
THE  PANEL  WERE  Uk.  JEROME  CORNFIELD,  AND  DR.  GEORGE  Y.  MANN). 


Ninth  Conference:  23-25  October  1963,  U.  S.  Army  Missile 
Contnand 

Professor  Solomon  Kullback 
COMMUNICATION  THEORV 

Professor  Frank  Proschan 

THE  CONCEPT  OF  MONOTONE  HAZARD  RATE  IN  SVSTEMS  REALI ABILITY 
Or.  Churchill  Elsenhart 

REALISTIC  EVALUATION  OF  THE  PRECISION  ANP  ACCURACY  OF 
INSTRUMENT  CALIBRATION  SVSTEMS 

Professor  H.  0.  Hartley 
NONLINEAR  ESTIMATION 

Professor  D.  B.  Ouncan 

ON  THE  SIMULTANEOUS  ESTIMATION  OF  A  MISSILE  TRAJECTORV 
AND  THE  ERROR  VARIANCE  COMPONENTS  INCLUDING  THE  ERROR 
POWER  SPECTRA  OF  SEVERAL  TRACKING  SVSTEMS 

Professor  Boyd  Harshbarger  (Chairman) 

PANE!  DISCUSSION  ON  WHAT  TVPE  OF  STATISTICIANS  ARE  NEEDED 
IN  RESEARCH  AND  DEVELOPMENT  LABORATORIES.  (OTHERS  ON  THE 
PANEL  WERE  DR.  E.  L.  COX,  DR.  CHURCHILL  EISENHART .  MR.  JOHN 
L.  MCDANIEL,  DR.  PAUL  R.  RIDER,  DR.  WILLIAM  WOLMAN  AND 
DR.  DONALD  A  GARDINER). 

Tenth  Conference:  4-6  November  1964,  The  Army  Research  Office, 
Washington,  DC 

MAJ  GEN  Leslie  E.  Simon  (Rat'd) 

THE  STIMULUS  OF  5<  S,  WILKS  TO  ARMV  STATISTICS 

Professor  Oscar  Kempthorne 

WtLOmNT  OF  THE  DESIGN  OF  EXPERIMENTS  OVER  THE  PAST  TEN 
VEARS 

Professor  H.  0.  Hartley  and  Professor  A.  W.  Wortham 
ASSESSMENT  AND  CORRECTION  OF  DEFICIENCIES  IN  PERT  ANALVSIS 
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Dr.  Churchill  Elsenhart 
SMI  WILKS  AS  T  REMEMBER  HIM 

Dr.  W.  J.  Youden 

AN  OPERATIONS  RESEARCH  VARH  A NO  OTHER  COMMENTS 

Professor  John  W.  Tukey 

THE  FUTURE  OF  PROCESSES  OF  DATA  ANALYSTS 

Dr.  M.  Q.  Kendall 
STATISTICS  AMD  MANAGEMENT 

Professor  Gerald  J.  Liebeman  (Chairman) 

PANE.'.  DISCUSSION  ON  RECRESSIOM  ANALYSIS.  (OTHERS  ON  THE  PANEL 
WERE  PROFESSORS  ROBERT  BECHHOFER,  0.  E.  P.  BOX,  JACK  C.  KIEFER 
AND  INGRAM  OLKIN) . 

Eleventh  Conference;  20-22  October  1965,  U.  S.  Army  Munition  Command, 
Dover.  Held  on  the  campus  of  Stevens  Institute 
of  Technology  In  Hoboken,  New  Jersey 

Dr.  Joan  R.  Rosenblatt 

CONFIDENCE  LIMITS  FOR  THE  REALIABIIITY  OF  COMPLEX  SYSTEMS 

Professor  J.  Stuart  Hunter 

NONLINEAR  MODELS*  ESTIMATION  AND  DESIGN 

Professor  William  C.  Guenther 
TARGET  COYERAGE  PROBLEMS 

Professor  H.  0.  Hartley 

MAXIMUM  LIKELIHOOD  ESTIMATES  FOR  THE  GENERAL  MIXED  ANALYSES  OF 
VARIANCE  MODEL 

Professor  R.  E.  Btchhofer 

PANEL  OTSCUSSTON  ON  SELECTING  THE  8E5T  TREATMENT.  (THE  OTHER 
PANEL  MEMBER  WAS  PROFESSOR  SHANTT  S.  GUPTA! . 
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Twalfth  Conftrence:  19-21  Cctobar  1966,  Harry  Diamond  Labs, 
and  the  National  Bureau  of  Standards 

Professor  Brian  W.  Conolly 
OPERATIONS  RESEARCH 

Or.  John  Handel 

STATISTICS  AS  A  DIAGNOSTIC  TOOL  IN  DATA  ANALySIS 

Professor  W.  <5.  Cochran 

PLANNING  AND  ANALYSIS  OF  OBSERVATIONAL  STUDIES 

Professor  Norman  L.  Johnson 
SAAIPLE  CENSORING 

Thirteenth  Conference:  1-3  November  1967,  The  U.  S.  Army  Mobility 
Equipment  Development  Center  and  the  U.  $. 
Army  Engineer  Topographic  Laboratories 

Professor  Francis  J.  Anseombe 
REGRESSION  ANALYSIS 

Professor  K.  A.  Brownlee 
SOME  C OMMCVTS  ON  MATCHING 

Professor  I.  J,  Good 

SOME  STATISTICAL  METHODS  IN  MACHINE  INTELLIGENCE  RESEARCH 
Dr.  Frank  Prosehan 

MAXIMUM  LIVELIHOOD  ESTIMATION  OF  REIIA8I LIT/ 

Dr.  M.  B.  Milk 
DATA  ANALySIS 

Fourteeeth  Confarenca;  23-25  Octobar  1968,  U.  S,  Army  Edgtwood 
Arsenal 

LT  GEN  William  B.  Bunker 

8R0ADENIN0  THE  H0RI20NS  OF  EXPERIMENTAL  DESIGN 

Professor  Rolf  E.  Bergmann 

STRUCTURE  AND  CLASSIFICATION  OF  PATTERN'S 


Professor  Acheson  J.  Duncan 
BULK  SAMPLING 

Professor  Emanuel  Pirzen 
TIME  SERIES 

Dr.  Walter  D,  Foster  (Chairman) 

PANEL  DISCUSSION  ON  BULK  SAMPLJNO,  (OTHERS  ON  THE  PANEL 
WERE  PROFESSORS  ACHESON  J.  DUNCAN  AM  SOW?  WARSH8AROER  ANC 
MESSRS.  HENRY  ELLNER,  GENE  RAY  LOURINORE,  JOSEPH  MANDELSON 
AND  VERNON  H.  RECHMEYER) . 

Fifteenth  Conference;  22-24  October  1969,  U.  S.  Army  Missile 
Command 

Dr.  John  E.  Condon 
RELIABILITY  A NO  Q.UA  L1TY  ASSURANCE 

Dr.  Nancy  R.  Mann 
SYSTEMS  RELIABILITY 

Dr.  Clifford  J.  Malonay 
A  PROBABILITY  APPROACH  TO  CATASTROPHIC  THREAT 

Profaisor  Richard  G.  Krutchkoff 

THE  EMPIRICAL  BAYES  APPROACH  TO  THE  DESIGN  AND  ANALYSIS  Of 
EXPERIMENTS 

Dr.  S.  C.  Saunders 

ON  CONFIDENCE  LIMITS  FOR  THE  PERFORMANCE  OF  A  SYSTEM  WHEN 
FEW  FAILURES  ARE  ENCOUNTERED 

Sixteenth  Conference;  21-23  October  1970,  U.  S.  Army  logistics 
Management  Center 

Professor  Solomon  Kullback 

MINIMUM  DISCRIMINATION  INFORMATION  ESTIMATION  AND  APPLICATION 

Dr.  Richard  0.  Kaplan 
FIELD  TESTING 
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Profaitor  Gary  G.  Koch 

THE  analysis  or  COMPLEX  contingency  table  data  prom  general 
EXPERIMENTAL  DESIGNS  ANP  SAMPLE  SURMEVS 

Proftiior  A.  Clifford  Cohan 

ESTIMATION  IN  TRUNCATED  POISSON  DISTRIBUTIONS  WITH  CONCOMITANT 
INTERVALS  ANP  TRUNCATION  POINTS 

Profaitor  Dana  Quad* 

NONPARAMETRIC  ANALYSIS  OF  COYARIANCE 

Savantaanth  Confaranca;  27-29  Oetobar  1971,  Waltar  Raad  Army 
Instltuta  of  Raiaareh 

Profaiaor  Marvin  Zalan 

THE  ROLE  OF  MATHEMATICAL  SCIENCES  IN  BIOMEPICAL  RESEARCH 
Profaiaor  8arnard  G.  Grarnbarg 

RANDOM IIEP  RESPONSE!  A  NEW  SURYEY  TOOL  TO  COLLECT  PATA  OF  A 
PERSONAL  NATURE 

Dr.  Gaof fray  H.  Ball 

CLASSIFICATION  ANP  CLUSTERING  nCHNJQilES  IN  PATA  ANALYSIS 

Profaitor  K.  S.  Banarjaa 
HOTELLING'S  WEIGHING  DESIGNS 

Dr.  John  J.  Gart 

THE  COMPARISON  OF  PROPORTIONS!  A  REVIEW  OF  SIGNIFICANCE  TESTS, 
CONFIDENCE  INTERVALS  ANP  ADJUSTMENTS  FOR  STRATIFICATION 

Elohtaanth  Confaranca;  25-27  Oetobar  1972,  U.  S.  Army  Tait 
and  Evaluation  Command 

Profaitor  John  Tutay 
EXPLORATORY  PATA  ANALYSIS 

Profaitor  G.  S.  Nation 
ORIENTATION  ANALYSIS 

Profaitor  J.  Stuart  Huntar 
SEQUENTIAL  FACTORIAL  ESTIMATION 


Profe»*or  6.  E.  P.  B** 
forecasting  ano  control 

Profmor  Raymond  H.  My*r* 

PUAL  RESPONSE  SERFAGE  ANALYSIS 

tmwfl  aaaa^'  EfiESSt 

mint  EnpInoHnfl  Trplnlnp  A9*ncy 

Profmor  J*rom<  Cornfltld 
8AVESIAN  STATISTICS 

PROCEDURES  FOR  MULTIVARIATE  NORMAL 

POPULATIONS 

Profmor  H.  L.  Gr*y 
GENERALIZES  JACKKNIFE  techniques 

Pro f««*or  Frtnk  Proichin 
RELIA8UITV  GROWTH 
Profmor  S.  C.  Sound* f« 

ACCELERATED  LIFE  TESTING 

Profmor  W.  A.  Thomp*on>  Op. 

RELIABILlTy  OF  MULTIPLE  COMPONENT  SVSTEMS 

Tw.nH.th  Canf«rwM.»  5J;?l*5*lMtl”%!n4  •"*“ 
EnfllBW  C»nt*r  *t  Ft.  BtWolr 

Or.  Churchill  El 

SAMUEL  S.  WILKS  AND  THE  ARMV  DESIGN  CONFERENCES 

Profmor  Solomon  Kulltwck 
MULTIDIMENSIONAL  CONTINGENCY  TABLES 
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Proftiior  Harbart  Solomon 
MULTIPARIANT  PATA  ANAl/SIS 

Proftiior  H,  A.  David 
ORPER  STATISTICS 

Proftiior  Gtrald  Lltbtrman 

reliability 

Proftiior  Robtrt  Btchhoftr 
RANKING  ANP  SELECTION  PROCEPURES 

On.  Marlon  R,  Bryion  tnd  WIT  Him  Mtllloi 
MAXIMUM  INFORMATION  FROM  EXPERIMENTS 


Twintv-FIrit  Con  fanner. 


22-24  Octobtr  1975.  Waltar  Raid  Amy 
Madlctl  Ctnttr  tnd  tht  Armtd  Forcti 
initltuta  of  Pathology 


Proftiior  Frtdarlck  Moitallar 

SUCCESS  IN  SOCIAL  ANP  MEPICAL  EXPERIMENTATION 

Proftiior  Edmund  A,  Gthtn 
NONRAWPOMIZEP  CLINICAL  TRIALS 

Proftiior  Paul  Malar 
RANPOMI2EP  CLINICAL  TRIALS 

Proftiior  Stymour  Balittr 
PREPICTIVE  SAMPLE  REUSE 

Proftiior  Edmond  Murphy 
NORMALITY  ANP  PISEASE 


Twntv-Stcond  Conftnscr  20 -ZJ^Oc tob-r  1 1  •  J6 .  Harry  Diamond  Labi. 

Profanor  0.  Stuirt  Huntar 
THE  MEASUREMENT  PROCESS 
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Professor  Benjamin  S,  Blanchard 
MANAGEMENT  OP  RELIABILITY 

Or.  Carl  N.  Morris 

STEWS  ESTIMATOR,  ITS  GENERALIZATIONS  AMO  ITS  APPLICATIONS 

Profaitor  Robert  V.  Hogg 
ON  ROBUST  STATISTICAL  PROCEDURES 

Profit aor  Noter  0.  Slngpurwalla 
ACCELERATED  LIFE  TESTING 

Twenty -Third  Confuranct;  19-21  Octobtr  1977.  U.  S.  Army  Combat 

Development*  Experimentation  Command.  Held 
at  the  Naval  Poitgriduate  School,  Monterey, 
California 

Professor  G.  E.  P.  Box 
TIME  SERIES  MODELING 

Professor  Norman  Breslow 
CENSORED  DATA 

Professor  Donald  P.  Gaver 

MODELING  AND  ESTIMATION  OF  COMPLEX  SySTEM  AVAILABILITY 

Professor  H.  0.  Hartley  (Keynote) 

ANALYSIS  OF  UNBALANCED  EXPERIMENTS 

Professor  Rupert  Miller 

THE  JACKKNIFE t  SURVEY  AND  APPLICATIONS 

Twenty-Fourth  Conference:  2-6  October  1978,  Mathematics  Research 

Center,  University  of  Ulsconsln-Madlson 

Professor  Ralph  Bradley 

SOME  APPROACHES  TO  STATISTICAL  ANALYSIS  OF  WEATHER  MODIFICATION 


Mr.  Bernard  Devi*  (for  Professor  Richard  E.  Barlow) 

RECENT  ADVANCES  IN  GRAPHICAL  TECHNIQUES  FOR  ANALYZING 
FAILURE  DATA 

Professor  Norman  Draper  (Keynote) 

RIDGE  REGRESSION 

Professor  8rtan  Joiner 
STATISTICAL  CONSULTING 

Professor  Grace  Wahba 

DESIGN  PROBLEMS  IN  RECOVERING  mCUOUS  OF  TWO  OR  SEVERAL 
VARIABLES 

Twenty-Fifth  Conference:  17-19  October  1979,  U.  S.  Army  Natick 

Research  and  Development  Command 

Professor  William  G.  Cochran  (Keynote) 

SUMMARIZING  THE  RESULTS  OF  A  SERIES  OF  EXPERIMENTS 

Dr.  Frank  E.  Grubbs 

A  QUARTER  CENTURV  OF  ARMV  DESIGN  OF  EXPERIMENTS  CONFERENCES 
Mr.  A1  l.  May 

DESIGNED  EXPERIMENTS  IN  SENSOR/  TESTING 
Dr.  Ray  E.  Schafer 

COMPUTER  AIDED  H/POTHESIS  TESTS  -  THE  BIRNBAUM  TEST 
Professor  Warren  Stewart 

NEW  ALGORITHMS  FOR  NONLINEAR  LEAST  SQJUARES  AND  8A/ESTAN 
•PARAMETER  ESTIMATION 

Professor  Marvin  Zelen 

ETHICS  AND  STRATECV  IN  THERAPEUTIC  INVESTIGATIONS 


SAMUEL  S.  WILKS  MEMORIAL  MEDAL 


The  Samuel  S.  Wilks  Memorial  Medal  Award,  Initiated  In  1964  by 
the  U.  S.  Army  and  Amarlean  Statistical  Association  Jointly,  It 
admlnl stored  by  the  American  Statistical  Association,  a  non-profit 
educational  and  scientific  society  founded  In  1839.  The  Wilks  Award 
Is  given  each  year  to  a  statistician  and  Is  basod  primarily  on  Ms 
contributions  to  the  advancement  of  scientific  or  technical  knowledge 
In  Army  statistics,  Ingenious  application  of  such  knowledge,  or 
successful  activity  In  the  fostering  of  cooperative  scientific 
matters  which  coincidentally  benefit  the  Army,  the  Department  of 
Defense,  the  U.  S.  Government,  and  our  country  generally. 

The  Award  consists  of  a  medal,  with  a  profile  of  Professor  Wilks 
and  the  name  of  the  Award  on  one  side,  the  teal  of  the  American 
Statistical  Association  and  name  of  the  recipient  on  the  reverse, 
and  a  citation  and  honorarium  related  to  the  magnitude  of  the  Award 
funds.  The  annual  Army  Design  of  Experiments  Conferences,  at  which 
the  Award  It  given  seen  year,  are  sponsored  by  the  Army  Mathematics 
Steering  Committee  on  behalf  of  the  Office  of  the  Chief  of  Research 
and  Development,  Department  of  the  Army. 

The  funds  for  the  S.  S.  Wilks  Memorial  Award  were  donated  by 
Philip  0.  Rust,  retired  Industrialist,  ThomatvIUa,  Georgia. 


RECIPIENTS  OF  THIS  AWARD 

NAME  DESIGN  CONFERENCE  (and  year)  PRESENTED 

Vn.  Paaak  E.  QiwJobi  Ttnth  ( 19*4 ) 

BatUitlc  RutoAch  Laboncutony 


Pit ojciio*  John  Tukiy 
Pninciton  Univvaity 

Union  G tMJtaJl  Le itli  E.  Simon 
United  Statu  A Amy  (Rat'd) 

Pnoiuton  W.  0.  Co chnon 
Ha wand  Uniw vulty 

Pnoituon  Jt mu  Utyran 
UniviMity  o{  CaUieniUa-BtAhUty 

Vn,  V.  2.  Youdtn 

national  Ionian  oi  StandaluU  (Rat'd) 


tliMtnth  ( 1965) 
TioUitk  ( 1966) 
ThJMi'vith  U9tt) 
fouAt  tenth  ( T96S] 
UittuUk  ( 1949) 
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PfiOieAion  Gtcxflt  W.  SnerftcoA 

Ioim  State  U*UveaiXtt/  (R®* 

T»w«ew«  R.  W.  M£ 

SS&5i(»»Si, 

PM^tiiO*  H.  0. 

Texa*  AIM  UiUve**^ 

Mx.  CuthbtAt  Pallet 
Co>au£tflrtt 

Pxaluiox  Htxbe'ut  SoZomoit 
Stanjoxd  UniuftMAtl/ 

p»rt([a4*ox  SotoifP«  Kufcfcbacli  . 

Gtoxjl  W<u/U*9*°K  UiUvtwxti/  (R«*  <0 

M.  CluMicWtf  E-UwtKdAt,  . 

Montana/  Buatau  Standard* 

Px^aaox  WbtW*J  {*•  1 Kau*^ 
Utvtwea*AtV  o<  ClUtaRo 


S-t itwUft  1  1970) 
SevewXccivtii  (><71 1 

E^glwtetn«i  ( 1^72) 
Nirttttta^l  (1973) 
TutmtiOh  (19741 
Tu'Ciitg'R'i^At  ((975) 
Tiveitty-Seccmd  ((971) 
Tw!fitUy-TIUA£(  1 197 1 ) 
TttH/vti/'Fouath  (1971) 
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AGENDA 


THE  TWENTY-FIFTH  CONFERENCE  ON  THE  DESIGN  OF  EXPERIMENTS  IN 
ARMY  RESEARCH,  DEVELOPMENT  AND  TESTING 

17-19  October  1979 

Host:  The  Natick  Research  and  Development  Command 

Held:  Auditorium,  Administration  Building,  Natick 
Research  Laboratories,  Natick,  Massachusetts 


*****  Wednesday,  17  October  ***** 


0815-0915 

0915-0930 


0930-1200 


0930-1030 


1030-1100 


REGISTRATION  —  Lobby,  Administration  Building 

CALLING  OF  THE  CONFERENCE  TO  ORDER  —  Lobby 

Donald  Kass,  Chairman  of  Local  Arrangements,  Natick 
Research  and  Development  Command 

WELCOMING  REMARKS 

COL  Robert  J.  Cuthbertson,  Natick  Research  and 
Development  Command 

GENERAL  SESSION  T. 

CHAIRMAN  -  Douglas  B.  Tang,  Chief,  Department  of  Bloetatistlcs/ 
Applied  Mathematics,  Walter  Reed  Army  Institute  of 
Research,  Waahington,  DC 

Special  Silver  Anniversary  AddresB 

A  QUARTER  CENTURY  OF  ARMY  DESIGN  OF  EXPERIMENTS 
CONFERENCES 

Frank  E.  Grubbs,  Program  Committee  Chairman,  Ballistic 
Research  Laboratory,  Aberdeen  Proving  Ground,  Maryland 

BREAK 
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1100-  ,1200 


GENERAL  SESSION  I  (Continued) 


1200-1330 

1330-1500 


1500-1530 

1530-1630 


1830-1930 


KEYNOTE  ADDRESS 

SUMMARIZING  THE  RESULTS  OF  A  SERIES  OF  EXPERIMENTS 

William  G.  Cochran,  Department  of  Statistics,  Harvard 
University,  Cambridge,  Massachusetts 

LUNCH 

TECHNICAL  SESSION  I  —  ROBUSTNESS  AND  OUTLIERS 

CHAIRMAN  -  Francis  E.  Dressel,  Army  Research  Office, 
Research  Triangle  Park,  North  Carolina 

APPLICATION  OF  ROBUST  FILTERING  AND  SMOOTHING  TO 
TRACKING  DATA 

William  S.  Agee  and  Robert  H.  Turner,  Analysis  and 
Computation  Division,  White  Sands  Missile  Range, 

New  Mexico 

ROBUST  REGRESSION  ANALYSIS  IN  PREDICTING  CERAMIC 
STRUCTURAL  FAILURE 

Donald  M.  Neal,  US  Army  Materials  &  Mechanics  Research 
Center,  Watertown,  Massachusetts 

COMPOUND  FREQUENCY  DISTRIBUTIONS 

Donald  W.  Rankin,  US  Army  Materiel  Test  &  Evaluation 
Directorate,  White  Sands  Missile  Range,  New  Mexico 

BREAK 

GENERAL  SESSION  II 

CHAIRMAN  -  Donald  Kass,  Natick  Research  and  Development 
Command,  Natick,  Massachusetts 

ETHIC i  AND  STRATEGY  IN  THERAPEUTIC  INVESTIGATIONS 

Marvin  Zelen,  Department  of  Statistical  Sciences, 

Harvard  School  of  Public  Sciences,  Boston,  Massachusetts 

SOCIAL  HOUR  —  Maridon  Restaurant 


1930 


BANQUET  —  Maridon  Restaurant 


*****  Thursday,  18  October  ***** 


0900-1030 


1030-1100 

1100-1200 


1200-1330 

1330-1530 


TECHNICAL  SESSION  II  —  STATISTICAL  THEORY 

CHAIRMAN  -  Grady  Miller,  US  Army  Materiel  Systems 

Analysis  Activity,  Aberdeen  Proving  Ground, 
Maryland 

ON  SIMULTANEOUS  INFERENCE  FOLLOWING  A  SIGNIFICANT 
KRU SKAL- WALL I S  TEST 

Andrew  P.  Sons,  Mathematics  Research  Center,  University 
of  Wisconsin-Madison 

SOME  BAYESIAN  ALTERNATIVES  TO  SIGNIFICANCE  TESTING 

Thomas  Leonard,  Mathematics  Research  Center,  University 
of  Wisconsin-Madison 

VARIANCE  REDUCTION  IN  MONTE  CARLO  SIMULATION 

Mark  Brown,  Florida  State  University;  Herbert  Solomon, 
Stanford  University;  and  Michael  A,  Stephens,  Simon 
Fraser  University,  Burnaby,  B.C. 

BREAK 

TECHNICAL  SESSION  III  —  DATA  ANALYSIS 

CHAIRMAN  -  Carl  dates,  US  Army  Concepts  Analysis  Agency, 
Bethesda,  Maryland 

SENSITIVITY  OF  TOLERANCE  LIMITS  TO  SMALL  SAMPLE  SIZES 

James  R.  Knaub,  Jr.,  US  Army  Materiel  Test  and  Evaluation 
Directorate,  White  Sando  Missile  Range,  New  Mexico 

THE  PERIODIC  NATURE  OF  EXPERIMENTALLY  MEASURED  DATA 

Michael  Hacskaylo,  Night  Vision  and  Electro-Optics 
Laboratory,  Ft.  Belvoir,  Virginia 

LUNCH 

GENERAL  SESSION  III 

CHAIRMAN  -  Edward  Robb,  Natick  Research  and  Development 
Command,  Natick,  Massachusetts 

DESIGNED  EXPERIMENTS  IN  SENSORY  TESTING 

Alfred  T.  May,  Pillsbury  Research  Labs,  Minneapolis, 
Minnesota 


1530-1600 

1600-1700 


0600-0900 


0900-0930 


COMPUTER  AIDED  HYPOTHESIS  TESTS  -  THE  BIRBAUM  TEST 

Ray  E.  Schafer,  Hughes  Aircraft  Company,  Fullerton, 
California 

BREAK 

TECHNICAL  SESSION  IV  —  LEAST  SQUARES 

CHAIRMAN  -  Maxson  Greenland,  Chemical  Systems  Laboratory, 
Aberdeen  Proving  Ground,  Maryland 

VARIABLE  TRANSFORMATIONS  IN  NONLINEAR  LEAST  SQUARES 
PROBLEMS 

Alvars  Celmlns,  Ballistic  Research  Laboratory,  Aberdeen 
Proving  Ground,  Maryland 

ANALYSIS  OF  DATA  WITH  THE  NONLINEAR  LEAST  CHI  SQUARE 
ALGORITHM 

Richard  L.  Moore,  US  Army  Armament  Research  and  Development 
Command,  Dover,  New  Jersey 


*****  Friday,  19  October  ***** 

TECHNICAL  SESSION  V  —  BIOMEDICAL  APPLICATIONS 

CHAIRMAN  -  William  E.  Baker,  Probability  and  Statistics 

Branch,  Ballistic  Research  Laboratory,  Aberdeen 
Proving  Ground,  Maryland 

ERROR-TIME  RESPONSE  PERFORMANCE  OF  NAIVE  SUBJECTS 

Michael  Hacskaylo  and  Joseph  E.  Swistak,  Night  Vision 
and  Electro-Optics  Laboratory,  Ft.  Belvoir,  Virginia 

IMAGE  INTERPRETATION  PERFORMANCE  IN  FOUR  STANDARD  TY^ES 
OF  AEROGRAPHIC  FILM 

Ronald  L.  Johnson,  US  Army  Mobility  Equipment  Research 
and  Development  Command,  Ft.  Belvoir,  Virginia 

GENERAL  SESSION  IV 

CHAIRMAN  -  Frank  E.  Grubbs,  Ballistic  Research  Laboratory, 
Aberdeen  Pro  ing  Ground,  Maryland 

OPEN  MEETING  OF  THE  AMSC  SUBCOMMITTEE  ON  PROBABILITY 
AND  STATISTICS 

Douglas  B.  Tang,  Chief,  Department  of  Biostatistics/Applied 
Math,  Walter  Reed  Army  Institute  of  Research,  Washington, 


xxv  i  I  i 


0930-1000 


BREAK 


1000-1100  GENERAL  SESSION  IV  (Continued) 

NEW  ALGORITHMS  FOR  NON-LINEAR  LEAST  SQUARES  AND  BAYESIAN 
PARAMETER  ESTIMATION 

Warren  Stewart,  Mathematics  Research  Center,  University 


of  Wlsconsin-Madison 

1100  ADJOURN 

******* 

*  *  * 

Carl  Bates 

PROGRAM  COMMITTEE 

Walter  Foster 

J.  Richard  Moore 

George  E.  P.  Box 

Frank  E.  Grubbs 

Douglas  Tang 

Larry  Crow 

Donald  Kass 

Malcolm  Taylor 

Franc is  Dressel 

Robert  Launer 

Michael  White 
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SPECIAL  SILVER  ANNIVERSARY  ADDRESS:  A  QUARTER 
CENTURY  OF  ARMY  DESIGN  OF  EXPERIMENTS  CONFERENCES 
Frank  E.  Grubb* 

Ballistic  Research  Laboratory,  Aberdeen  Proving  Ground,  Maryland 

Welcome  to  the  2Sth  Conference  on  the  Decign  of  Experiments 
in  Army  Research,  Development  and  Testing.  A  whole  quarter  of 
a  century  has  passed  since  Sam  Wilks  recommended  that  the  Army 
start  this  series  of  conferences,  and  what  an  excellent  idea  he 
had,  with  all  the  vision  for  the  future  of  Army  statistics.  As 
we  all  know,  Wilks  was  a  very  remarkable  man:  a  gentleman,  a  good 
leader,  an  outstanding  scholar  and  research  statistician,  a  man 
who  also  had  very  vital  interests  in  applications,  and  he  liked  to 
see  people  work  together.  Sam  travelled  much  for  the  Department 
of  Defense  and  he  consulted  widely  on  all  probable  areas  of  sta¬ 
tistical  application  for  the  Government.  He  missed  none  of  these 
conferences,  and  we  remember  him  so  well  in  his  role  of  selecting 
many  of  the  key  statisticians  in  the  universities  to  participate 
in  these  conferences,  as  we  met  at  the  Cosmos  Club,  in  Washington, 
and  drank  and  dined  with  Sam.  I  think  we  have  a  better  pay-off 
from  these  statistical  conferences  than  the  other  DOD  conferences, 
because  of  the  close  interface  with  university  statisticians,  in¬ 
cluding,  of  course,  the  eminent  statistician  who  gives  the  keynote 
address  next.  To  the  memory  of  Sam  Wilks  we  owe  so  much,  and 
therefore  in  1964  we  devoted  the  Army  Design  of  Experiments  Con¬ 
ferences  to  Sam's  memory. 

This  particular  conference,  the  landmark  25th,  is.  also  dedica¬ 
ted  to  our  good  friend,  fellow  mathematician  and  statistician, 
teacher,  and  excellent  administrator,  Dr.  Francis  G.  Dressel.  How 
would  the  design  of  experiments  conferences  ever  have  survived  if 


it  had  not  been  for  Francis  ?,  who  carried  always  the  big  part  of 

the  load.  I  am  asking  him  to  make  a  few  remarks  next.  I  am  glad 

Francis  has  trained  Bob  Launer  so  well  tool 

Now  how  in  the  world  did  Z  ever  get  saddled  with  a  "Special 

Silver  Anniversary  Address",  including  the  fancy  title  that  came 

from,  I  might  say,  a  former  friend?  It  is  probably  beoause  some 

of  my  colleagues  saw  me  enjoying  beii^g  too  much  a  "free  lancer" 

at  these  conferences,  so  they  thought l  I  must  stretch  the  exact 

title  of  my  talk  a  bit  to  cover  more  statistics.  Back  in  the  mid- 

Thirties,  I  was  a  timid,  very  illiterate  Southerner,  trying  to  eke 

out  a  living  by  teaching  engineering  math  at  (now)  Auburn  University, 

and  it  became  starkly  clear  that  we  lowly  instructors  would  hardly 

ever  be  promoted  unless  we  got  &  Ph  Dl  But  it  was  also  made  quite 

clear  to  us  that  getting  a  Ph  D  would  not  make  us  a  better  teacher l 

A  quick  OB  study  (not  so-called  then)  convinced  us  that  we  should 

.  as  an  outlet, 

seek  something  in  applied  math,  A and  statistics  was  the  subject  to 
study,  for  it  was  needed  and  spreading  fast  too,  for  example  to  our 
agricultural  experiment  station  problems.  Z  had  a  friend,  who  got 
a  Ph  0  at  Cornell  University,  and  had  &  good  job  at  our  experiment 
station,  so  that  he  introduced  me  to  the  analysis  of  variance, 
which  seemed  to  be  a  misnomer,  and  ha  even  alarmed  me  with  the 
idea  of  the  analysis  of  covariancel  I  later  heard  that  Karl  Pearson 
was  the  greatest  statistician  of  all,  but  that  one  R.  A.  Fisher  was 
not  sprinkling  holy  water  on  all  the  things  Karl  Pearson  had  done. 

My  friend  wanted  to  know  if  Z  had  read  any  of  the  reputable  jour¬ 
nals  on  the  subject  of  statistics,  and  I  hadn't,  of  course.  In 
fact,  in  about  1934  a  paper  by  a  young  genius  on  the  distribution 
of  quadratic  forms  in  a  normal  system,  with  applications  to  the 
analysis  of  covariance  had  appeared  in  a  (strange)  journal  called 


the  Proceedings  of  the  Cambridge  Philoaophoical  Society,  written 
by  our  keynote  speaker  of  today.  Later,  in  the  early  1940's  when 
I  had  been  m  uniform  at  the  BR1  during  World  War  XX,  we  had  trouble 
justifying  a  journal  called  "Biometrika"  at  first,  and  there  was  no 
way  the  Army  would  ever  approve  the  Annals  of  Eugenics  in  our  Li¬ 
brary  even  if  it  contained  statistical  papers  1 

Xr.  any  event,  back  in  the  mid-  to  late  30 's  it  did  seem  that 
statistics  would -be  a  good  choice  of  graduate  study  with  an  out¬ 
let.  But  where  in  the  United  States  could  one  study  statistics? 

He  certainly  could  not  do  so  down  South,  and  in  fact  there  were 
only  two,  or  maybe  three,  places  to  go  for  statistics  courses  -  Iowa 
State  University,  the  University  of  Iowa,  and  perhaps  the  University 
of  Michigan.  The  latter  was  mostly  an  acturial  school  with  well- 
known  voices  such  as  Menge  and  Glover,  with  Cecil  C.  Craig  and 
Paul  Dwyer  coming  along,  and  perhaps  most  interesting  of  all  an. 
athletic  statistician  and  actuary,  Harry  Clyde  Carver,  who  would 
challenge  his  graduate  students  to  beat  him  at  any  sport  of  their 
own  choice.  If  Carver  won,  there  would  be  a  stiff  final  exam  and 
no  A' si  One  had  to  beat  Carver  at  his  own  game! 

We  selected  Michigan,  for  Xowa  State  University  seemed  too 

far  away,  and  Alan  T.  Craig  of  Xowa  was  scheduled  to  give  the 

basic  graduate  statistics  course  at  Michigan  in  the  summer  of  1937, 

and  what  a  good  start  to  learn  to  throw  dice,  et  all  That  summer, 

I  tried  to  learn  what  a  random  variable  was.  I  had  known  Clifford 

Cohen  for  years  back  at  Auburn,  and  at  Michigan,  Clifford  of  all 

things,  had  elected  to  write  his  dissertation  on  the  very  obscure 

subject  of  truncated  sample  theory  1  But  how  in  the  world  could 

there  ever  be  much  interest,  let  alone  wide  applications,  of  such 

an  odd  topic?  A  colleague,  who  had  treaded  this  mill  before, 

assured  me  that  "Clifford  Cohen  was  a  vary 

3 


smart  man  -  he  chose  a  topic  no  one  else  would  ever  work  on I" 

The  advice  continued,  "Don't  ever  pick  a  'hot'  topic,  for 

a  genius  will  beat  you  to  it,  and  you'll  never  get  your  degree" l 

.in  mathematics 

The  non-statistical  graduate  students  /  frowned  at  anyone  studying 
statistics,  as  it  was  not  as  important  as  topology  or  even  dif¬ 
ferential  geometry  either.  One  of  them  was  somewhat  friendly 
though,  and  on  occasion  would  drop  by  our  "flat"  with  his  wife. 

He  would  pick  up  and  continue  my  wife's  needlepoint  with  much  en¬ 
thusiasm,  but  didn't  care  for  statistios  then,  or  any  part  of  itt 
His  name  -  Jimmy  Savage  { l ) .  And  it  goes  to  show  you  what  can  hap¬ 
pen  toat  pure,  rigorous  mathematician,  once  he  is  "bitten  by  the 
bug"  or  otherwise  the  clever  ideas  of  the  Reverend  Thomas  Bayes t 

It  was  not  easy  to  to  find  and  settle  upon  a  dissertation  topic 
without  some  guidanae,  but  all  the  professors  already  had  too 
many  graduate  students,  and  thsy  had  passed  along  topics  to  some 
they  never  heard  from,  so  that  I  had  "batter  look  around  in  the 
library".  No  one  then  told  me,  for  example,  that  concerning  trun¬ 
cated  sample  theory  this  would  develop  into  the  field  of  order 
statistics,  and  moreover  blossom  into  reliability  ,  life-testing, 
et  al,  and  it  was  in  fact  many,  many  years  before  that  did 
occur.  You  see,  no  advice  I  had  been  given  really  sunk  in,  for 
1  decided  to  work  on  outliers,  and  the  international  situation 
had  gotten  so  gloomy  that  writing  a  dissertation  would  not  be 
done  very  quickly  anyway,  So,  being  a  reserve  officer,  I  was  intro- 
to  the  Army. 

The  physical  and  engineering  scionces  were  just  beginning  to 

make  some  uses  of  statistics,  although  Walter  Shewhart  had  made 

>at  Michigan 

applications  of  statistical  quality  control.  In  1941  '  I  got  ahold 
of  a  new  book,  "An  Engineers's  Manual  of  Statistical  Methods"  by 

one  Major  Leslie  E.  Simon,  and  as  I  read  it  and  was  enlightened 
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by  the  book,  some  correspondence  developed,  for  here  was  an 
authority  who  had  the  vision,  the  wisdom  and  the  courage,  of 
all  things,  at  the  time  of  e  very  low  cycle  on  the  Bayesian  revo¬ 
lution  (so  that  we  later  kidded  Les  Simon  that  he  had  the  unmiti¬ 
gated  gall)  to  publish  in  the  back  of  his  book  some  Charts  to 
estimate  the  fraction  of  defectives  in  a  lot  by  using  Bayes* 
equally  likely  hypothesis!  Today,  there  seem  to  be  no  100% 

classical  statisticians,  so  that  we  can  chalk  up  another  win  for 
* 

Les.  And  this  introduced  me  to  the  Army's  Ballistic  Research 

Laboratories,  on  active  duty  in  uniform  as  a  Lieutenant,  where 

there  was  never  to  be  an  end  to  all  kinds  of  knotty  statistical 
problems. 

As  pointed  out  in  Les  Simon's  book,  An  Engineers  Manual  of 
Statistical  Methods,  Dr.  L.  S.  Dederick  had  worked  out  the  proba¬ 


bility  distribution  of  the  sample  range  (largest  minus  smallest 
observation)  back  in  1926,  and  had  partially  tabulated  its  dis¬ 
tribution,  but  wouldn't  submi€  it  for  publication!  Sam  Wilks  had 
on  occasion  consulted  with  the  person¬ 
nel  of  the  Ballistic  Section  of  BRL  at  Aberdeen  Proving  Ground 

on  various  statistical  problems.  Also,  since  the  dispersion  of 
shots  on  a  target,  as  from  rifle  firings,  was  often  measured  by 
the  "extreme  spread",  or  bivariate  runge,  this  little  nasty  sta¬ 
tistical  distribution  had  eluded  statisticians.,  and  Mr.  Philip 
G.  Rust,  an  industrialist  and  "rifle  accuracy  bug",  established 
by  sampling  shot  patterns  che  distribution  of  the  extreme  spread 

for  small  sample  sixes.  Also,  on  the  train  from  Washington  to 
Wilmington,  Phil  Rust  had  told  Sam  Wilks  about  it  and  had  suggested 

that  he  look  into  the 

*  In  a  panel  discussion  on  Bayesian  methods  on  reliability  one  time,  I  statsd  that 
statistically  I  vas  501  classical.  25%  fiducial  and  25%  Bayesian,  but  Frank  Proschan 
promptly  branded  me  as  a  hermaphrodite. 
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theory  of  the  probability  distribution  of  the  extreme  spread  in 
order  to  study  its  properties,  as  it  was  widely  used  in  ballis¬ 
tics  and  rifle  accuracy  competititons  also. 

When  I  arrived  at  the  BRL  at  Aberdeen  Proving  Ground  in  1941, 
John  von  Neumann,  Robert  H.  Kant,  H.  R.  Beilinson,  and  B.  I.  Hart 
had  just  worked  out  and  published  in  the  Annals  of  Mathematical 
statistics  the  distribution  of  the  mean  square  successive  differ¬ 
ence,  and  the  mean  square  successive  difference  to  the  variance, 
and  B.  X.  Hart  had  calculated  percentage  points  of  both. 

The  "real  world  data",  coming  out  of  ballistic  testing  of  all 
kinds,  often  defied  any  good  or  "normal"  analysis,  and  were  loaded 
with  outliers l  There  was  thus  an  applicable  dissertation  topic! 
in  the  mid  Forties,  there  existed  a  critical  need  to  do  something 
about  speeding  up  the  production  of  firing  tables,  as  about  100 
female  "computers"  were  always  busy  running  computations  on  those 
big,  heavy  desk  electrio  Friden  or  Monroe  calculators. 

Leslie  E.  Simon  valued  brains  to  solve  the  Army's  problems  in 
ballistics,  and  he  had  established  a  scientific  advisory  committee 
with  some  of  the  best  brains  in  the  physical  sciences  in  the 
country.  What  a  wonderful  and  stimulating  place  to  work,  less  much 
time  to  be  in  uniform  and  fight  the  "battle  of  Aberdeen" l 

Back  in  the  early  Thirties,  Simon  was  Chief  of  Manufacture 
at  Picatinny  Arsenal,  and  had  cultivated  the  interest  and  exper¬ 
tise  of  Walter  Shewhart  to  apply  the  principles  of  statistical 


at  Picatinny  Arsenal, 
quality  control  to  the  manufacture  of  ammunition'  At  the  BRL 

in  World  War  11 ,  Simon  saw  the  pressing  need  to  pass  on  the 

principles  of  statistical  quality  control  to  industry  in  the 

production  of  ammunition  and  weapons  for  the  US  Army,  and  he 

also  worked  with  Harold  Dodge  of  the  Bell  Telephone  Laboratories 

to  start  computations  of  the  standard  sampling  inspection  tables 

for  the  Army  Ordnance  Corps,  later  put  into  Military  Standard  105A. 

Sam  Wilks  had  long  been  aware  of  the  need  for  well-designed 
experiments  and  hence  suggested  that  the  Army  start  a  series  of 
annual  conferences  to  promote  statistical  methods.  Sam  suggested 
that  the  Design  of  Experiments  Conferences  should  have  three  types 
of  sessions:  First,  there  would  be  some  special  invited  papers 
by  well-known  authorities  on  the  philosophy  and  general  principles 
of  statistical  design  of  experiments,  than  there  would  be  some 
technical  papers  presented  by  Army  statisticians, ■ and  finally 
there  would  be  clinical  sessions  with  suggestions  from  the  experts  - 
and  we  still  stick  to  this  format  today.  These  conferences  had 
their  beginning  19-21  October  1955  at  the  Diamond  Ordnance  Fuze 
Laboratory  and  National  Bureau  of  Standards  in  Washington,  D.  C. 

We  note  that  Sam's  conferences  were  Army  Wide,  and  attracted  DOD 
interest,  while  a  conference  the  Ballistic  Research  Laboratories 
put  on  a  year  earlier  (1954)  on  the  use  of  statistical  methods 
was  primarily  for  Army  Ordnance  personnel. 

Within  Ordnance  and  the  Army,  Leslie  E.  Simon  certainly  was 

i 

the  great  stimulus  to  the  advancement  of  statistical  methods,  for 
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at  the  BRL  Les  was  not  only  its  Director,  but  he  also  prepared 
a  large  number  of  papers  on  engineering  statistics  or  statistical 
engineering  -  what  ever  you  want  to  call  it.  Moreover,  there  was 
a  pressing  need  for  these  very  papers  to  acquaint  industry  to 
the  methods  of  quality  control  and  statistics  in  connection  with 
the  World  War  II  effort.  And  Les  helped  promote  the  short  courses 
on  statistical  methods  in  industry.  There  was  a  great  deal  o£ 
interest  during  this  period  concerning  the  concept  of  "economical 
lot  sizeB",  and  also  the  concept  of  producing  very  large  "homo¬ 
geneous  lots"  so  that  for  ammunition  at  least  we  could  get  rid  of 
the  situation  where  at  a  field  artillery  battery  site  there  existed 
a  mixture  of  rounds  from  several  or  many  lots  with  different  levels 
of  muzzle  velocity  and  degrees  of  surface-finish  roughness. 

In  the  mid-nineteen  Forties,  a  very  significant  and  World- 
Wide  development  occurred  due  to  an  idea  of  our  imminent  and  es¬ 
teemed  Scientific  Advisory  Committee  member,  John  von  Neumann.  He 

had  suggested  the  construction  of  the  EN1AC  or  Electronic  Numerical 

a  digital  computer  - 

integrator  and  Calculator  '  at  the  BRL.  We  saw  the  handwriting 
on  the  wall:  The  ENIAC  could  be  used  to  Monte  Carlo  anything  to 
death,  in  addition  to  the  more  straight-forward  .computations  of 
mathematical  and  statistically  tractable  functions.  And  so  many 
statistical  problems  were  planned!  First,  however,  there  had  to 
be  some  calculations  on  the  distributions  of- outliers,  and  tho 
ENIAC  staff  was  looking  for  work!  It  was  then  that  I  learned 
about  priorities  and  the  real  importance  of  any  statistical  problem 
to  the  country! 
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Once  they  got  the  ENIAC  wired  for  my  outlier  problem,  the  Atomic 
Energy  Commission  called  on  Gen  Simon  and  Johnny  von  Neumann  to 
use  the  ENIAC  to  obtain  an  optimum  solution  to  the  problem  of 
imploding  the  core  of  a  nuclear  warhead,  with  the  result,  of 
course,  that  even  though  our  Computing  Laboratory  had  begged  for 
work,  my  suggestion  and  the  start  of  some  extensive  calculations 
to  keep  them  busy  immediately  got  a  vanishing  priorityl 

In  those  days  of  a  great  scientific  effort  at  the  BRL  and 
many  of  the  Country's  great  physicists,  chemical  physicists,  mathe¬ 
maticians,  engineers,  at  al,  I  felt  just  like  Les  Simon  had  always 
said,  "The  engineers  would  call  him  a  statistician,  and  the  statis¬ 
ticians  would  call  him  an  engineer"'!  Indeed,  we  were  trying  to 
apply  statistical  techniques  to  many  knotty  physical  problems  for 
which  there  was  a  phisical  model  that  applied  well.  Perhaps 
I  would  have  been  much  better  off  in  agriculture!  We  survived 
some  way  or  the  other  and  hence  got  away  from  the  use  of  primarily 
the  probable  error  which  wae  never  to  be  deleted  from  firing  tables 
though 1 

Now,  getting  back  to  the  uses  of  Army  Statistics,  which  led 
up  to  the  Design  of  Experiments  Conferences,  we  record  that  a 
very  good  aocount  of  the  statistics  in ‘ the  Army  from  the  very  be¬ 
ginning  has  been  prepared  by  our  good  friend  Clifford  Maloney  (The 
American  Statistician,  June  196?) ,  who  traced  various  statistical 
interests  in  one  way  or  the  other  from  very  early  times  -  he 
started  out  with  Daniel  Bernoulli  in  1777.  As  pointed  out  by 
Maloney,  there  certainly  was  much  vital  interest  in  medical 
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statistics  of  the  Army  from  the  beginning,  and  at  West  Point  the 
graduates  needed  to  know  something  about  the  dispersion  of  shots 
on  a  target,  and  least  squares  and  the  adjustment  of  data.  In 
fact,  in  early  times,  the  best  engineers  in  the  USA  were  really 

i 

coming  from  West  Point.  Then  again,  there  was  always  considerable 
interest  in  the  sensitivity  of  explosives  to  impact  or  shock,  the 
sensitivity  of  primers  and  other  items  of  Ammunition,  which  no 
doubt  brought  about  the  so-called  "Bruceton  Method"  of  sensitivity 
analysis,  and  later  developed  into  the  Dixon-Mood  "Up  and  Down" 
technique  at  Princeton,  and  since  has  been  widely  used.  Naturally, 
Dixon  and  Mood  were  students  of  Sam  Wilks,  who  again  enters  the 
general  picture  1  So  you  see,  the  Army  did  indeed  have  the  most 
natural  needs  and  demands  for  the  application  of  statistical 
methods,  and  Sam  Wilks  was  the  first  university  professor  to  recog¬ 
nize  this  vital  development  for  the  good  of  all  concerned,  as  he 
was  always  in  touch  with  so  many  important  applications. 

I  think  that  the  Army  Design  of  Experiments  (DOE)  Conferences 
Sam  Wilks  started  have  performed  the  the  vital  task  of  fulfilling 
the  need  for  cross-fertilization  of  statistical  theory  and  prac¬ 
tice,  even  though  these  conferences  occurred  only  once  a  year. 

It  is  through  the  Army  DOE  conferences  that  we  have  become  ac¬ 
quainted  with  each  other,  discussed  common  statistical  problems, 
presented  solutions  to  others,  learned  a  lot  from  the  eminent 
university  statisticians  and  gotten  their  best  suggestions  during 
the  clinical  sessions.  Moreover,  this  has  all  stimulated  Army 


satisticians  to  perform  very  good  work  and  publish  a  number 
of  useful  results  for  others  to  apply,  if  it  had  not  bean  for 
Wilks'  vision  and  the  DOE  conferences,  we  would  have  been  off 
to  ourselves, no  doubt, working  very  much  more  inefficiently. 

Now  all  of  you  have  the  nice  little  pamphlet  prepared  by  Bob 
Launer  for  this,  the  Twenty-fifth  anniversary  of  the  Army  DOE 
conferences,  and  we  note  that  the  series  got  off  with  a  bang  in  1954 
aB  Bill  Cochran  led  with  the  philosophy  underlying  the  the  design  of 
experiments.  Churchhill  Eisenhart  spoke  on  principles  of  ran¬ 
domization  (Isn't  that  still  an  unsettled  topic  ?)  and  John  Tukey 
headed  a  panel  on  "Where  do  statisticians  fit  in?").  Jack  Youden 
aided  in  his  most  interesting  way  of  talking  about  the  design  of 
experiments  in  industrial  research  and  develpoment.  The  applica¬ 
tion  of  order  statistics  and  problems  in  subjective  testing  came 
into  the  Second  DOE  conference,  and  we  were  fortunate  to  have 
R.  A.  Fisher  at  the  Third  conference!  Also  at  the  Third  oonf ar¬ 
enas  Ho  Hartley  spoke  on  changes  in  the  outlook  of  statistics 
brought  about  by  modern  computers,  and  Ben  Epstein,  who  at  one 
time  even  worked  at  Frankford  Arsenal,  covered  what  was  to  become 
a  very  important  Army  field  "life-testing"  -  and  later  reliability 
and  reliability  growth.  Here  at  Natick  for  the  Fourth  conference, 
it  was  appropriate  to  have  L.  H.  C.  Tippett  disauss  statistical 
methods  in  the  textile  industry,  and  the  Fifth  conference  taught 
me  a  lot  about  smoking  and  lung  cancer  (now  forgotten.!)  because 
of  the  lively  debates  between  Joe  Berkson  of  the  Mayo  Clinic  and 
Jerzy  Neyman  (both  smoking,  X  believe!) 


The  theme  or  title  of  these  conferences  was  stretched  many 

times,  I  am  reminded,  to  include  many  important  topics  of  the 

day  or  time,  and  this  was  necessary  and  good  toot  In  fact,  I 

note  that  Egon  Pearson  gave  the  keynote  address  of  the  Eighth 

conference  on  a  statistician's  place  in  assessing  the  likely' 

« 

operational  performance  of  Army  weapons  and  equipment,  or  the 
need  for  statistics  in  military  operations  research  and  weapon 
systems  analysis.  In  fact,  the  Army  has  a  parallel  series  of 
conferences,  started  in  1961,  known  as  the  Army  Operations  Re¬ 
search  Symposia.  I  found  that  the  field  of  operations  research 
was  being  staffed  primarily  by  mathematicians,  physicists,  en¬ 
gineers  and  others,  but  not  enough  statisticians,  who  could  aid 
in  their  modelling  problems  of  stochastic  processes.  For  example, 
for  probability  of  hitting  problems  there  was  often  the  need  to 
have  simple  approximations  to  the  distribution  of  quadratic  forms 
in  normal  variables,  and  techniques  like  the  Wilson-Hilf erty 

transformation  of  Chi-square  to  approximate  normality  and  the 

.those  darn  that 

Polya-Wilson  approximation  to' cut-off  normal  integrals/ were  found 
to  be  very  useful.  Moreover,  we  also  saw  that  the  theories  of 
life-testing  would  apply  to  Lanchester  type  combat  theory. 

Because  of  the  critioal  need  for  the  evaluation  of  weapon  systems, 
and  later  many  other  military  operations  research  topics,  the 
Army  OR  symposia  have  attracted  a  large  number  of  "high  brass" 
type  visitors.  Statistical  topics  have  been  often  discussed  at 
the  Army  OR  conferences  and  OR  topics  at  the  Army  DOE  conferences. 
Forget  titlesl 


In  the  Spring  of  1964,  some  six  months  before  the  Tenth  con¬ 
ference,  we  got  the  shock  of  our  lives  with  the  untimely  passing 
of  Sam  Wilks.  For  the  Tenth  conference,  Les  Simon  came  forth  with 
an  excellent  and  informative  paper  on  the  stimulus  of  S.  S.  Wilks 
to  Army  Statistics,  and  the  high  importance  of  the  DOB  conferences 
to  Army  statistical  endeavors. 

Fortunately,  the  DOE  conferences  have  proceeded  to  cover  the  watez 
front,  and  stimulate  and  train  more  statisticians. 

Now  although  I  have  mentioned  many  of  the  key  benefits  and 
much  in  the  way  of  significant  progress  that  has  resulted  from 
the  DOB  conferences,  let  me  now  jump  to  a  look  at  the  whole  series, 
or  the  view  from  an  operations  research  eye.  To  begin  with, 
it  becomes  quite  clear  that  we  have  learned  a  lot  about  modelling 
processes  (stochastic)  or  fitting  models  to  data  in  order  to 
make  more  general  predictions,  or  to  summarize.  "Models"  ? 

Yes!  And  this  reminds  me  of  what  George  Box  is  quoted  as  saying, 

"All  models  are  wrong,  (but)  some  (even)  work" 1  How  true  this 
isl  Aren't  models  competitive,  and  haven't  we  found  that  the 
situation  doesn't  exist  for  which  only  one  model  is  right  and 
all  the  others  wrong?  In  fact,  we  are  often  lucky  that  any  of 
several  competitive  models  may  serve  the  purpose  at  hand  very 
well.  Yes,  I  think  we  have  learned  how  to  model  many  important 
Army  areas  of  application,  and  this  has  also  brought  about  model 
development  or  better  theories. 

Obviously,  the  great  benefit  to  the  Army  from  the  DOE 
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conferences  has  been  the  expert  counseling  of  in-house  statis¬ 
ticians  by  the  eminent  university  statisticians  who  have  so  kind¬ 
ly  given  of  their  time  and  experience.  One  has  only  to  look  at  the 

i 

little  booklet  of  featured  speakers  to  be  very  highly  impressed 
with  the  caliber  of  the  talent.  We  greatly  appreciate  this,  as 
their  help,  including  the  clinical  session  suggestions,  has  been 
outstanding,  and  for  very  difficult  areas  of  application. 

This  brings  to  mind  another  point.  The  US  Army  is  a  very 
large  and  diversified  organization.  In  case  you  need  some  con¬ 
verting  on  this  point,  just  attend  one  of  the  Army  Science  Con¬ 
ferences  held  biennally  at  West  Point.  In  addition  to  our  little 
corners  of  application  we  have  discussed  over  the  years,  at  the 
Army  Science  conferences,  they  have  presented  papers  on,  for  ex¬ 
ample,  sampling  the  polar  ice  caps  -  which  brought, up  many  sta¬ 
tistical  problems  of  note  -  or  even  the  extraction  and  analysis 
of  snake  venoml  What  are  the  main  controversies  about?  You 
guessed  it:  the  instrumentation,  the  measurements  and  their  in¬ 
terpretation.  Army  investigators  has  grown  increasingly  aware 
of  errors  of  measurement,  precision  and  accuracy,  and  even  just 
how  to  define  these  illusive  concepts.  And  so  have  others.  For 
example,  Committee  E-ll  of  the  American  Society  for  Testing  and 
Materials  has  for  some  20-25  years  been  working  on  the  problem 
of  standardizing  the  views  of  engineers,  chemists,  etc.,  on  the 
subjects  of  precision  and  accuracy,  and  come  forth  with  a  recom¬ 
mended  practice.  I  still  don't  see  an  end  to  this  effort,  for 


there  are  enough  "divinely  endowed",  stubbornly  statistical  minds 

to  bring  about  nothing  but  impasse's,  (incidentally,  l  know  that  I 
alone  am 

right  though,  and  they  needn't  think  they  can  sway  me  to  a  com¬ 
promise!).  As  a  passing  remark,  there's  a  full  time  job  for  a 
young,  competent  statistician  for  NASA,  the  FAA,  and  such  agencies, 
in  connection  with  sampling  the  atmosphere  in  order  to  establish 
temperature  profiles,  ozone  content  profiles,  etc.,  by  knowing  the 
capability  of  their  instrumentation  for  the  first  time, 

we  have  learned  much  about  the  statistical  design  and  analysis 
of  scientific  type  experiments,  and  the  construction  of  designs  - 
the  latter,  I  think!  Furthermore,  I  see  evidence  very  frequently 
of  some  "fancy"  experimental  designs  that  Army  investigators  are 
using,  with  very  sophisticated  analyses,  too.  On  the  front  cover 
of  the  program  of  the  Tenth  Conference,  there  is  a  10  x  10  Graeco- 
Latin  Square,  and  no  one  yet  has  pointed  out  an  error  in  it! 

When  the  nice,  balanced  experiments  have  been  violated  in  one 
way  or  the  other,  speakers  like  Hoh  Hartley  (who  regrets  that 
his  duties  as  President  of  ASA  keeps  him  away  today)  have  come 
along  to  help  or  straighten  us  out.  We  have  used  linear  models 
mostly,  but  have  been  hit  by  nonlinear  models  at  times,  and 
George  Box  has  on  several  occasions  given  us  his  unique  approach 
to  time  series  analysis. 

There  have  been  many  advances  over  the  years  in  the  analysis 
of  contingency  tables,  and  count  data  generally.  We  have  had 
many  contributors  on  this  subject  speak  to  uo,  and  the  several 


approaches  presented  to  us,  including  Kullback's  information 
theory  approach,  and  some  of  the  recent  work  of  Peinberg  in 
our  preceding  two-day  tutorial  course.  Perhaps  the  US  Army 
Operational  Test  and  Evaluation  Agency  has  made  much  use  of 
contingency  table  analyses,  and  have  benefitted  from  them. 

I  will  continue  to  try  and  sort  out  that  problem,  and  I  note 
that  my  experience  has  been  primarily  in  connection  with  the 
comparison  of  two  or  more  binomial  type  proportions,  and  irre¬ 
spective  of  Fisher's  fixed  marginals,  and  stuff  like  that,  I 
still  don't  want  to  confuse  the  issue  by  imbedding  the  compari¬ 
son  of  binomial  p's  in  a  contingency  table  analysis.  Maybe  the 
real  experts  have  other  views. 

Hasn't  the  field  of  reliability  and  related  applications 
hit  us  with  a  big  bang,  to  say  the  least  ?  And  the  high-level 
"brass"  or  managers  have  shown  the  greatest  of  interest  in  it 
too.  Remember,  I  remarked  that  Clifford  Cohen  in  the  late  1930 's 
v:rote  his  dissertation  on  the  obscure  subject  of  truncated  sample 
theory?  Well,  finally  the  area  came  to  life  and  how!  Although 
the  normal  distribution  was  the  "universal"  one  in  the  past,  it 
didn't  "take"  with  the  reliability  analysts  at  all,  and  they 
aimed  for  the  exponential  distribution.  At  tho  1977  Monterey 
Conference  (23rd),  a  paper  was  given  by  Herback,  Green  and  Blumen- 
thal  on  the  "curse"  of  the  exponential  model,  and  they  quotes 

"The  exponential  is  wrong, 

But  works  like  a  song. 

Beware  the  Weibull: 

It's  incorrigible"  -  Anon 

Remember  George  Box  -  All  models  are  wrong! 


16 


Thera  is  some  heavy  interest  in  reliability  growth,  and  the 

and 

whole  field  of  reliability  will  continue/ will  continue  to  expand. 
There  are  now  so  many  methods  or  recommendations  for  obtaining 
confidence  bounds  on  system  reliability  that  an  appointed  cotamittee 
has  not  been  able  to  standardize  on  a  technique  for  DOD.  It 
might  be  said  that  sample  order  statistics  are  of  much  importance 
to  the  Army  nowadays,  and  often  even  help  to  take  care  of  the 
outlier  problem.  Finally,  reliability  analysts  have  worked  on 
estimation  and  other  properties  of  the  two-  and  three-  parameter 
Weibull  models  so  much  that  this  has  actually  aided  in  the  spread 
of  the  Weibull  distribution  to  many  other  areas  of  application 
than  reliability.  Perhaps  this  is  because  of  the  robustness  of 
the  Weibull  model  in  representing  a  variety  of  shapes. 

A  very  old  statistical  problem  is  that  of  bio-assay  types  of 
analysis,  and  it  borders  on  the  estimation  of  risks  and  safety 
levels  in  any  number  of  other  fields.  There  are  many  papers  on 
the  subject  of  quantal  response,  "sensitivity  analyses",  explo¬ 
sive  sensitivity  (Bruceton) ,  ballistic  limit,  Up  and  Down  method,  etc 
which  have  been  aired  in  these  conferences.  Quantal  response 
investigations,  and  especially  the  estimation  of  both  high  and 
low  percentage  points  (of  unknown  distributions),  does  indeed 
cover  a  very  important  statistical  effort  for. the  Army,  and  it 
will  continue  to  expand  also.  Maybe  this  is  an  area  for  which 
the  use  of  physical  models  is  needed  in  addition  to  statistical 
analyses,  or  at  least  a  combination  of  both.  We  will  face  more 


and  more  safety  type  problems,  for  which  there  will  be  heavy  de¬ 
mands  for  statistical  treatment,  while  we  used  to  avoid  them  al¬ 
together.  Let  me  mention  one  nasty  little  problem  having  to  do 
with  armor  protection  in  tanks,  or  penetration  mechanics,  and 
hence  for  safety  of  tank  crews  as  a  result  of  armor  thickness 
determination  and  design.  It  is  also  a  statistical  problem  the 
Army  continues  to  need  help  on  to  estimate  the  parameters  for 
zero  chance  of  penetration.  Do  you  like  continuous  distributions 
that  slowly  change  to  a  series  of  binomial  and  continuous  models 
of  some  kind?  And  even  approach  a  binomial  distribution  with 

parameter  zero?  In  this  case,  we  start  firing  at  a  piece  of  armor 

plate  of  a  certain  thickness,  and  for  the  high-striking  velocities 

we  will  (usually)  get  100%  penetrations  of  the  projectiles  through 

the  plate,  and  there  will  be  a  "residual"  velocity  distribution 

for  the  projectiles  or  pieces  of  projectiles  which  have  penetrated 
and  aome  off  the  back. 

the  plate  But  as  the  striking  velocity  is  decreased,  then  the 
proportion  of  projectiles  penetrating  the  plate  will  decrease, 
ultimately  to  zero  for  low-striking  velocities,  and  thus  we  say 
that  a  safety  level  exist  somewhere,  or  at  least  we  would  like  to 
know  just  where,  for  example,  only  1  in  a  1000  of  the  projectiles 
would  penetrate.  The  curve  or  residual  velocity  versus  striking 
velocity  gets  very  steep  near  the  bottom,  obviously,  and  its  a 
challenge  to  ballisticians  and  statisticians  to  deal  with  the 
precise  and  highest  striking  velocity  for  which  zero  penetrations 
occur.  What  I  am  also  indicating  is  that  there  are  many  problems 
of  interest  for  which  statisticians  and  physical  scientists  must 
work  as  team  members,  and  the  DOE  conferences  guarantee  just  that. 
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Let  us  not  forget  the  field  of  sampling  inspection  or  aoceptanoe 
sampling  inspection,  and  the  DOD's  use  of  standard  sampling  in¬ 
spection  tables  and  practioea.  These  are  important  activities 
that  the  Army  initiated  with  the  original  help  of  Harold  Dodge* 
and  our  DOE  Proceedings  include  a  number  of  papers  on  the  subject. 
This  is  really  the  area  of  statistics  that  taught  us  much  about 
operating  characteristic  curves,  or  power  curves,  and  the  deter¬ 
mination  of  sample  sizes,  and  the  like.  Thus,  many  statistical 
areas  of  interest  spill  over  into  other  topics,  and  so  the  prooess 
continues. 

Now  I  have  made  my  little  choices  of  some  of  the  benefits 
and  topics  of  value  that  we  have  been  priveleged  to  be  part  of 
in  our  twenty-five  years  of  Army  Design  of  Experiments  Conferences. 
Perhaps  you  can  expand  or  Improve  on  what  I  have  covered  and  hence 
make  more  sense  out  of  things.  I  invite  you  to  do  so.  In  any 
event,  it  certainly  seems  quite  clear  that  these  conferences 
have  been  very  "cost-effective"  to  the  Army. 

I  think  it  was  Cliff  Maloney  who  once  suggested  that  there 
should  be  published  a  volume  of  the  best  papers  of  the  proceedings 
of  these  conferences.  This  assignment  I  was  given  made  me  look 
through  the  whole  shelf-wide  proceedings,  and  I  agree  that  there 
are  certain  of  the  papers  whiah  indeed  should  be  brought  together 
in  some  kind  of  memoirs. 

Maybe  we'  can  now  get  Francis  Dressell  to  make  a  remark  or 
two,  as  we  have  dedicated  this  the  landmark  25th  Conference  to 
him.  Francis  1 
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<rU5  - 

SUMMERIZING  THE  RESULTS  OF  A  SERIES  OF  EXPERIMENTS 
William  G.  Cochran 

Professor  of  Statistics  Emeritus,  Harvard  University 

I  first  met  this  problem  In  the  thirties  In  agriculture.  I  wrote  a 
paper  on  It  (1),  and  later  a  more  ambitious  paper  with  Yates  (2),  In  which 
a  number  of  examples  were  worked.  W&  trlee  to  see  In  what  respects  the 
analysis  of  a  group  of  experiments  resembled  and  In  what  respects  It  differed 
from  the  analysis  of  a  single  evpr*r1ment. 

The  need  to  summarize  results  of  a  series  of  experiments  on  the  same 
treatments  arises  In  two  types  of  application.  The  first  type  may  be  des¬ 
cribed  as  exploratory;  a  number  of  experiments  on  the  relative  performance 
of  something  or  of  two  treatments  have  been  carried  out,  and  we  are  trying 
to  answer  the  question;  what  Is  the  present  state  of  knowledge  about  the 
relative  merits  of  the  two  treatments?  For  Instance,  the  recent  academy 
study  of  saccharin  started  with  the  experiments  In  which  large  doses  were 
given  to  rats;  these  were  the  prime  experiments.  To  cite  a  second  example, 
Yates  and  Crowther  realized  at  the  beginning  of  World  War  II  that  Britain 
would  have  to  Import  most  of  her  fertilizers  during  the  war  and  would  be 
short  of  fertilizers.  Accordingly,  they  summarized  the  experiments  (4) 
about  the  responses  of  the  common  farm  crops  to  fertilizers  In  order  to 
answer  the  question:  What  Is  the  present  state  of  knowledge  about  the 
effects  of  fertilizers  and  to  provide  material  for  an  Intelligent  rationing 
system  for  fertilizers? 

As  another  example,  I  was  In  a  group  that  studied  two  common  methods  of 
surgery  for  duodenal  ulcer-vagotomy  (cutting  the  vagus  nerves)  plus  a  radi¬ 
cal  antrectomy  (which  removes  the  lower  portion  of  the  stomach)  versus 
vagotomy  plus  the  milder  pyloroplasty  (which  widens  the  outlet  of  the  stomach 
to  provide  better  drainage). 
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We  found  four  experiments  that  appeared  to  have  been  carefully  done 
and  properly  randomized.  We  could  have  come  across  a  number  of  comparisons 
that  were  well  done  but  not  randomized— the  type  sometimes  called  observa¬ 
tional  studies.  Since  often  we  cannot  use  randomization  and  have  to  make  a 
comparison  without  It,  I  would  have  been  Interested  In  Including  the  obser-  , 
vatlonal  studies  so  as  to  learn  whether  they  agreed  with  the  randomized 
studies  and  If  not,  why  not?  But  the  medical  members  of  our  team  had  been 
too  well  brought  up  by  statisticians,  and  refused  to  look  at  anything  but 
randomized  experiments.  In  this  type  of  surgery,  we  may  expect  the  experi¬ 
ment  to  be  of  different  designs  and  perphaps  differing  numbers  of  replications. 

The  second  type  of  application  occurs  commonly  In  agriculture.  It 
differs  from  the  first  In  two  ways.  It  Is  known, that  the  relative  perfor¬ 
mance  of  a  treatment  (variety  of  a  crop  or  fertilizer)  Is  likely  to  vary 
both  from  field  to  field  within  a  year  and  from  year  to  year.  Thus  experi¬ 
ments  are  likely  to  be  repeated  in  different  fields  and  for  a  number  of  years. 
Secondly,  there  Is  a  better  chance  that  the  experiments,  being  jointly  planned, 
are  of  the  same  design  and  number  of  replications.  For  Instance,  when  the 
growing  of  sugarbeets  was  Introduced  into  Britain  after  World  War  I,  the 
government  conducted  3x3x3  factorials  (ultimately  30  per  year)  at  the  leading 
centers  for  a  number  of  years. 

The  objective  of  the  experiments  may  be  a  series  of  decisions  as  to  which 
varieties  of  a  crop  look  promising  and  should  be  kept  for  further  testing, 
which  varieties  should  be  discarded,  and  which  varieties  having  been  fully 
tested,  should  be  part  of  an  approved  list  and  have  their  seed  made  available 
to  farmers.  As  an  example,  Patterson  and  Sllvey  (5)  have  described  the  trials 
of  varieties  of  cereals  that  Britain  has  conducted  In  recent  years,  the 
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designs  being  incomplete  blocks.  This  kind  of  screening  program  is  not 
confined  to  agriculture.  It  may  be  used  in  seeking  the  best  drugs  or 
vaccines  for  acme  purpose  in  medicine,  or  in  seeking  persons  best  capable 
of  doing  sane  task.  In  1963,  Federer  gave  a  bibliography  of  some  500 
papers  on  screening  programs. 

2.  Miscellaneous  Experiments  in  Exploratory  Work 

I’ll  start  with  exploratory  experiments  done  by  different  people  at 
different  places  and  times.  Since  these  experiments  were  not  planned  as 
a  coordinated  series,  we  must  expect  them  to  differ  in  designs,  and  in 
numbers  of  replications.  First  we  must  think  of  the  question:  of  what 
population,  if  any,  can  these  experiments  be  considered  something  approaching 
a  random  sample?  Is  this  population  relevant  to  future  applications  of  any 
conclusions  that  we  draw?  In  seme  oases  we  may  reluctantly  conclude  that 
the  experiments  do  not  sample  any  population  of  interest  to  ub,  arid  decide 
not  to  prepare  any  stannary.  In  same  oases  the  experiments  are  so  variable 
that  sane  must.be  thrown  out  before  any  summary  is  attempted.  The  way  in 
which  the  experiments  l,were  done  also  affeots  the  nature  of  the  population 
that  they  sample.  The  nature  of  the  experiments  also  affects  the  kind  of 
population  that  they  sample.  In  the  National  Academy  study  of  saccharin 
to  which  I  referred,  the  doses  in  the  laboratory  experiments  were  so  large 
that  the  estimates  of  the  effects  of  more  normal  dosera  depended  to  a 
substantial  extent  on  the  kind  of  model  used  In  extrapolating  the  experi¬ 
mental  results.  In  experiments  comparing  two  methods  of  surgery,  the 
experiments  may  be  confined,  for  ethical  or  logical  reasons,  to  the  kind 
of  patients  whose  doctors  state  beforehand  that  they  can  safely  take  either 
method  of  surgery.  Otherwise,  it  is  difficult  to  interpret  the  results  of 
the  experiments.  This  restriction  affects  the  character  of  the  population 
to  whan  the  conclusions  apply. 

In  agriculture,  as  I  have  stated,  we  have  to  contend  with  variations 
in  both  space  and  time:  But  in  other  fields  of  application  there  may  be 
no  strong  reasons  to  consider  time  as  a  separate  source  of  variation,  even 
though  the  experiments  will  have  presumably  been  done  at  different  times. 

So  in  considering  a  surrcnary  of  miscellaneous  exploratory  experiments,  I 
shall  combine  time  and  spaoe  and  speak  of  treatments  x  places. 
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experiments  Interactions  for  the  1th  Treatment  In  the  jth  experiment.  We 
may  also  expect  experiments  to  have  different  variances  o2  per  observation 
and  to  differ  In  number  of  replications  nj. 

For  the  jth  experiment,  a  model  that  seems  reasonable  with  a  quantita¬ 
tive  response  is  that  tha  mean  of  the  1th  Treatment  In  the  jth  experiment  Is 

*  u  +  *i  *  +  5u 

where  y^j  Is  the  treatments  x  experiments  Interaction  and  the  variance  of 
the  error  term  e^  Is  cr^/n^  (1  -  1,2,. ...t;  j  ■  1.2,. ...k). 

In  a  combined  analysis  of  these  means,  a  reasonable  first  step  Is  to 
form  a  two-way  treatments  x  experiment  table  of  these  means.  If  all  treat¬ 
ments  are  present  In  all  experiments,  an  analysis,  of  variance  Into  the 
following  components  should  be  easy. 


df 

Experiments 

(k-1) 

Treatments 

(t-1) 

Treatments  *  Experiments 

(t-l)(k-l) 

Pooled  error 

The  purpose  Is  to  test  the  Interaction.  If  some  treatments  are  missing  from 

some  experiments,  a  least  squares  analysis  appropriate  to  missing  data  Is 

used.  In  this  case  the  Treatments  line  Is  Treatments,  adjusted  for  experiments. 

The  pooled  error  In  the  analysis  of  variance  of  the  treatment  means  Is 

(l/k)(s2/n1j),  or  if  the  $2  seem  to  be  homogeneous,  s2(l/n.).  We  will  want 

2  J 

to  examine  whether  the  Sj  appear  to  be  heterogeneous,  since  this  affects  the 
F-test  of  the  ratio  treatments  x  experiments/pooled  error.  For  this  we  can 
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use  Bartlett's  test,  or  If  the  data  seem  nonnormal  and  we  want  a  more  robust 

test,  we  can  use  Levene's  test,  based  on  the  absolute  value  of  the  deviations 

that  lead  to  the  s  ,  that  appears  to  be  less  affected  by  ncnnonnality.  If  the 
2 

s  seem  markedly  heterogeneous,  the  P-test  of  the  Interactions  against  the  pooled 
error  Is  not  exact,  but  assingning  a  number  of  df  to  the  pooled  error  by 
Satterthwalte's  approximation  should  provide  an  approximate  test. 

The  next  step  Is  to  reach  one  of  three  decisions  about  the  Treatments  x 
experiments  Interactions.  (1)  that  It  Is  negligible,  (11)  that  It  Is  not 
negligible  but  has  no  dlscernable  structure.  By  this  I  mean  that  although 
the  effects  of  the  treatments  vary  from  experiment  to  experiment,  we  have 
no  Information  for  making  different  predictions  In  different  parts  of  the 
population  arid  must  draw  single  overall  conclusions  about  the  effects  of  the 
treatments&ujhe  third  case  Is  that  In  which  the  Interaction  Is  of  a  nature 
that  we  think  we  understand,  and  Is  large  enough  so  that  different  treatments 
win  In  different  parts  of  the  population  that  can  be  described.  In  this 
case  we  expect  to  recommend  different  treatments  for  different  parts  of  the 
population. 

Consider  first  case  (1)  In  which  we  judge  that  the  treatments  *  experi¬ 
ments  interactions  are  negligible.  If  the  experiments  differ  In  number  of 
replications  and  In  their  error  variances,  a  question  to  be  considered  Is: 

Should  the  treatment  means  1u  Individual  experiments  be  weighted  In  forming 


the  overall  means,  so  as  to  give  more  Influence  to  the  more  accurate  experl- 

2 

ments?  If  so,  what  should  the  weights  be?  If  the  error  variances  o ,  were 

2  J 

known,  the  weights  should  presumably  be  Wj  ■  Oj/nj,  but  the  variances  are 
only  estimated,  unless  the  Oj  appear  to  be  equal  so  that  weights  n^  can  be 


used  for  the  treatment  In  the  jth  experiment.  Various  authors  have  worked  on 
this  problem  of  weighting  with  fallible  weights. 

The  first  step  Is  to  find  out  if  there  Is  much  gain  In  accuracy  from 

2 

the  use  of  weighted  means.  If  the  Sj  appear  to  be  homogeneous,  and  the 
weights  are  the  known  values  nj/sj,  this  can  be  done,  because  the  ratio  of 

the  variance  of  the  weighted  to  the  unweighted  mean  of  the  Is 

(Iwj ) (Z  Vwj)/k  .  For  Instance,  If  one-third  of  the  experiments  each  have  nj 

with  relative  values  1,  1/2,  1/4,  the  relative  value  of  the  variance  of  the 
weighted  to  the  unweighted  mean  Is  36/49  -  0.73.  The  situation  Is  less 
favorable  to  the  weighting  if  the  Sj  differ,  so  that  we  have  to  use  something 
like  estimated  weights  h^/Sj.  Under  normality ,  the  maximum  likelihood  estimate 
of  the  overall  mean  is 


£ 

J 


Vj  +  ij  “  V 


o  . 


This  har  to  be  found  Iteratively.  In  this  type  of  estimate,  an  experiment 
2 

with  low  Sj  and  apparently  high  precision  Is  prevented  from  dominating  the 
overall  mean  if  11  disagrees  markedly  from  the  value  suggested  by  the  other 
experiments,  since  the  term  nj(y^j  -  u^)  will  be  large,  and  will  decrease 
the  weight  given  to  this  experiment. 

Some  years  ago,  C.  R.  Rao  (7)  brought  out  a  new  method  of  estimating 
variances  and  variance  components  called  the  MINQUE  (minimum  norm  quadratic 
unbiased  estimator).  Since  I  have  been  Interested  In  this  problem  for  over 
40  years,  I  asked  O.N.K.  Rao  of  Carleton  University  and  P.S.R.S.  Rao  of  the 
University  of  Rochester  If  the  MINQUE  method  woulu  lead  to  Improved  estimates 
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of  the  weighted  mean.  Both  men  looked  Into  the  problem— J.N.K.  Rao  In  the 
case  with  no  treatments  x  experiments  Interactions  which  Is  now  being  con¬ 
sidered  and  P.S.  Rao  In  the  case  In  which  we  assume  a  random  treatments  * 

2 

experiments  Interaction  with  variance  which  also  has  to  be  estimated. 
Both  men  discovered  what  I  had  suspected  In  working  with  MINQUE— that  If  one 
Is  trying  to  produce  an  Improved  method  of  estimating  variance  components, 

It  may  not  be  wise  to  make  the  estimates  unbiased.  With  unbiased  methods 
one  may  get  variance  component  estimates  that  sometimes  take  negative  values 
and  have  large  variances.  Both  men  produced  adjustments  to  MINQUE  that  are 
essentially  positive.  J.N.K.  Rao's  method  (8)  uses  non-iterative  weights 
rather  similar  to  the  maximum  likelihood  weights.  The  weights  are 


“j  ■  nj<fj 4  4  "jtfij  - 


where  y^  Is  the  unweighted  mean  of  the  y^.  Some  limited  Monte  Carlo  studies 
have  shown  that  the  weighting  does  better  than  the  maximum  likelihood  estimates 
of  the  treatments  means  except  when  differences  in  the  error  variances  are 
extreme.  This  estimate  also  does  better  than  MINQUE  and  better  than  the 
simple  weights  Wj  »  n./Sj  and  is  probably  the  best  found  thus  far. 

For  estimating  the  gain  In  accuracy  from  the  use  of  erroneous  weights 
like  these,  the  previous  figures  for  the  relative  accuracy  of  weighted  to 


unweighted  means  must  be  reduced,  because  of  sampling  error  In  the  weights. 
The  dampening  factor  depends  both  on  the  average  df  with  which  sj  are  esti¬ 
mated,  and  on  the  amount  of  heterogeneity  In  the  weights.  For  the  previous 
example  with  weights  proportional  to  1,  1/2  and  1/3  In  thirds,  and  1.36  If 
the  weights  are  known,  the  dampening  factor  Is  approximately  (T  +  6^?  +  8)  , 
where  f  is  the  average  number  of  df-in  Sj. 


2 

Thus  if  the  Sj  have  6  df  on  the  average,  the  relative  efficiency  of  weighted 
to  unweighted  means  is  estimated  as  (12)(1.36)/(14)  ■  1.18— a  rather  modest 
gain  from  weighting.  Before  resorting  to  weighting,  check  also  that  weighted 
means  apply  to  the  same  population  as  unweighted  means.  For  example,  If  the 
weights  tend  to  be  high  when  the  mean  yields  of  the  experiments  are  also 
high,  we  may  conclude  that  the  results  for  weighted  means  apply  to  a  popula¬ 
tion  having  a  higher  mean  yield  than  our  actual  population  and  decide  not  to 
use  weighted  means. 

For  comparison  between  the  estimated  means  of  the  treatments,  we  need 

standard  errors.  With  unweighted  means,  the  estimate  of  their  standard  error 
— 2 - 

Is  A(Sj/nj}/k.  With  the  experimental  error  variances  of  the  individual 
experiments  taken  as  homogeneous,  the  estimated  variance  of  the  mean  weighted 
as  nj  Is  s  /Enj.  For  Rao's  estimate  with  fallible  weights,  Rao  (8)  has  given 
a  rough  estimate  of  the  variance  of  this  weighted  n^an,  which  also  Implies 
a  dampening  factor  for  the  fact  that  fallible  weights  are  being  used.  Die 
jacknife  estimate  is  another  possibility,.. 

When  the  treatments  x  experiments  Interaction  is  significant,  we  need 
to  see  if  we  can  understand  the  nature  of  the  Interaction.  For  this,  a  two- 
way  treatments  x  experiments  table  of  residuals  Is  helpful.  Sometimes  there 
is  no  winner;  different  treatments  appear  to  win  In  different  parts  of  the 
population,  but  either  we  do  not  fully  understand  the  interaction  or  do  not 
wish  to  use  It  in  a  recommendation.  Sometimes  there  are  two  distinguishable 
parts  of  the  population  in  which  the  ranking  of  the  treatment  Is  different, 
and  we  understand  why.  Student  (10)  cites  an  example.  After  a  long  series 
of  experiments,  the  Irish  Department  of  Agriculture  Introduced  Spratt-Archer 
barley  as  the  best  silted  to  the  country.  In  one  county  the  farmers  '  efused 

?8 


to  grow  It,  claiming  that  their  native  barley  was  superior.  In  order  to 
convince  those  fanners,  the  Department  of  Agriculture  made  some  special 
comparisons  In  this  county  of  the  native  barley  versus  Spratt-Archer .  To 
their  surprise,  the  native  barley  was  superior.  The  reason  also  became 
clear.  This  barley  Is  a  quick-starting  variety.  Now  in  this  county, 
fanning  Is  rather  lackadaisical,  so  that  the  weeds  flourish.  The  weeds 
tended  to  smother  the  Spratt-Archer  barley,  which  starts  slowly,  but  ';he 
native  barley,  starting  quickly,  could  smothar  the  weeds.  Another  maxim 
from  this  example  Is  make  sure  the  experiments  sample  the  population  to  which 
their  results  will  be  applied. 

If  there  are  two  parts  which  have  kj  and  k2  experiments,  the  following 
breakdown  of  the  Interaction  Is  relevant 

df 

Treatments  (Part  I  -  Part  II)  (t-1) 

Treatments  Part  I  experiments  (t-lMk^-l) 

Treatments  Part  II  experiments  (t-l)(k2-l) 

In  this  breakdown,  we  expect  the  first  term  to  be  large  and  the  other  parts 
small.  In  addition,  we  need  to  analyze  parts  I  and  II  separately,  In  order 
to  see  If  there  are  definite  treatments  differences  In  each  part. 

If  the  Interaction  Is  significant  and  Is  assumed  to  be  random,  the 

2  2 

variance  of  a  treatment  mean  In  an  Individual  experiment  Is  (o^  +  a  /n^), 

2 

which  moves  nearer  equality  because  of  the  term  but  also  means  that  an 
extra  parameter  has  to  be  estimated  If  weighted  means  are  contemplated.  In 
a  Monte  Carlo  study  by  P.S.R.S.  Rao,  Kaplan,  and  Cochran  (9)  several  types 
of  weighted  means  Including  a  revised  MINQUE  were  Included  but  the  unweighted 
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p 

mean  proved  very  hard  to  beat,  as  might  be  expected,  unless  is 

2 

small  and  the  variation  in  the  Oj  is  extreme.  Use  of  the  unweighted 
mean  has  the  advantage  that  an  unbiased  estimate  mean  of  the  variance 
of  the  overall  mean  of  a  treatment  is  -  y)  /k(k-l) . 

If  the  original  observations  are  in  proportions,  remember  that  a 
decision,  e.g.  whether  a  single  overall  mean  has  enough  advantage  overall 
over  the  other  means  to  recommend  it,  or  whether  two  means  should  be 
recommended  for  different  parts  of  the  population,  must  be  made  in 
proportions.  If  the  combined  analysis  is  made  in  some  other  scale, 
such  as  angles  or  logits,  because  it  is  thought  nearer  to  normality  or 
in.  seme  ways  more  suitable,  remember  that  means  in  the  original 
proportional  soale  will  be  slightly  biased  when  we  transfer  baok. 
Quenoullle  (11)  ha3  given  approximate  corrections  for  this  bias,  which 

p 

do  not  appear  to  be  well  known.  Let  s  be  our  estimate  of  the  varianoe 
of  z  (where  z  denotes  the  transformed  soale),  that  is,  the  mean  in  the 
transformed  scale.  If  an  angular  transformation  is  used,  Quenoullle 's 
correctlon  for  bias  in  the  transformed  mean  is  to  increase  sin  z  by 
Js(l  -  e~2z  )cos  (2z).  If  logits  are  used  with  equal  weights,  the 
usual  procedure  is  to  take  p  -  ez  /  (1  +  ez)  when  transforming  back 
to  p.  Quenouille's  correction  for  bias  is  to  add  (n-l)s  /2n  to  z  before 
talcing  ez  /  (1  +  ez). 

3.  Variations  in  Both  Time  and  Space 

This  situation  is  likely  to  occur  primarily  in  agriculture.  Since 
the  experiments  are  likely  to  be  jointly  planned,  they  may  have  the  same 
designs  and  number  of  replications,  the  same  experiment  being  repeated 
at  the  same  place  for  four  or  five  years.  As  mentioned,  the  number  of 


30 


years  will  commonly  be  limited  to  at  most  four  or  five,  since  a  larger 
number  slows  up  any  reccirmendations.  But  the  experiments  may  not  have 
the  same  numbers  of  replication  -  more  may  have  been  added  in  later  years. 
In  varietal  trials,  a  new  variety  may  be  added  in  the  second  or  third 
years,  so  that  different  treatments  may  have  different  numbers  of  years 
at  any  given  time.  However,  unless  the  numbers  of  replications  differ 
greatly,  a  preliminary  analysis  of  the  treatment  means  will  usually  be 
adequate  and  is  fairly  easy,  although  there  are  extra  complications  and 
ftill  least  squares  may  have  to  be  used  if  seme  treatments  are  only  present 
in  the  later  years. 

It  will  usually  be  necessary  to  treat  the  treatments  x  years 

2 

variation  as  random,  with  variance  ,  even  if  it  does  not  act  like 

a  random  variate.  A  good  deal  is  known  about  the  influence  of  weather 

on  crops,  and  we  may  have  found,  for  Instance,  that  in  a  good  year 

the  best  treatments  have  a  greater  advantage,  so  that  the  treatments  x 

years  interaction  is  definitely  not  random.  But  a  superior  treatment 

before  recommendation,  must  be  superior,  on  the  average,  over  a  span 
2 

of  years,  taking  o^.  into  account,  since  we  cannot  recommend  a  different 
treatment  for  different  years. 

Ihe  preliminary  analysis  of  variance  and  the  expected  values  of  the 
means  squares  are  shown  below.  I  have  treated  the  treatments  x  places 
interactions  as  randan  as  well  as  treatments  x  years,  since  this  is 
usually  the  assumption  that  has  to  be  made  if  it  is  a  question  of 
recommending  the  overall  use  of  one  treatment. 
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df 

Expected  v< 

Treatments 

(t-1) 

-2  2 

0  *Mta> 

T  x  Y 

(t-l)(y-l) 

-2  2 

0  +  natpy 

T  x  P 

(t-D(p-i) 

-2  2 

0  +  notpy 

T  x  P  x  Y 

(t-l)(p-l)(y-l) 

-2  2 

0  +  notpy 

Pooled  error 

a2 

+  nya. 


In  presenting  the  expected  values,  I  have  taken  the  simplest  case, 

in  which  all  experiments  are  of  the  same  size  and  design,  the  symbols  n, 

t,  p,  and  y  standing  for  number  of  replications,  number  of  treatments, 

-2 

number  of  places,  and  number  of  years.  The  symbol  a  is,  of  course, 
the  true  pooled  error  variance.  The  MS^py  is  against  error,  and  if 

P  is  about  1,  this  mean  square  may  be  combined  with  the  pooled  error. 

The  expected  values  are  written  as  if  treatments  are  also  randan,  with 

p 

variance  o^,.  If  the  effects  of  treatments  are  fixed,  as  they  usually  are, 

P  P  2 

replace  by  what  is  usually  called  -  E(t-€)  /(n-1). 

From  the  expected  values  it  is  clear  that  the  treatments  x  years 

and  treatments  x  places  interactions  are  tested  by  an  appropriate  P  test 

p 

(approximate  if  Oj  varies  fran  experiment  to  experiment)  against  the  mean 
square  for  the  tpy  three-factor  interaction,  and  that  an  unbiased  estimate 
of  c£y  is  (MSty  -  MS^yVnp.  For  the  main  effects  of  Treatments,  no 
single  line  in  the  analysis  of  variance  is  a  proper  error.  An  unbiased 
estimate  of  the  error  variance  for  the  error  of  a  treatment  mean,  if 
interactions  are  present  and  random,  is 


MS.  +  MS.  -  MS. 
tp  ty  tpy 
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and  an  approximate  F  test  of  the  treatments  mean  square  may  be  made 
by  taking  F  ■  MSt/(MStp  +  MS^  -  MStpy) ,  with  Satterthwalte's 
approximation  used  to  ascribe  a  number  of  df  to  the  denominator. 

However,  In  a  small  Monte  Carlo  study  of  experiments,  Hudson  and 
Krutchkoff  (13)  found,  somewhat  surprisingly,  that  a  rival 
F  -  (MSt  +  MStpy)/(MStp  +  MSty)  using  Satterthwalte ,  had  somewhat 
better  power  and  reoontnended  it,  although  it  did  not  approximate  the 
5%  and  1J6  levels  of  F  when  the  null  hypothesis  was  true. 

Since  whether  we  reoommend  one  treatment,  two  treatments  or  suspend 
Judgement  for  seme  reason  depends  mainly  on  how  the  treatments  vary  in 
effects  from  plaoe  to  plaoe,  the  two-way  table  of  treatments  and  places 
deserves  careful  study.  The  treatments  x  places  interaction  is 
sometimes  heterogeneous;  some  comparisons  of  some  treatments  have  a 
higher  mean  square  interaction  than  others.  Subdivisions  of  the 
treatments  and  places  and  the  treatments  x  places  sum  of  squares  should 
be  tried. 

Thus,  as  we  have  seen,  the  summary  of  a  series  of  experiments  oalls 
mainly  for  experience  In  the  analysis  of  variance,  which  we  now  have. 

It  is  well  to  adopt  something  of  the  attitude  in  exploratory  analysis 
and  be  on  the  lookout  for  anything  unexpected, since  the  nature  of  the 
tp  Interaction  is  often  a  hard  thing  to  puzzle  out. 
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ABSTRACT 

Robust  mathods  provide  a  fresh  approach  to  the  problem  of  treatment  of 
wild  observations  In  filtering  and  smoothing  problems.  The  robust  M-estl- 
mates  of  regression  are  exteiidad  to  filtering  and  fixed  lag  smoothing  em¬ 
ploying  a  pseudo-density  of  the  observations  In  a  conditional  mean  deriva¬ 
tion  of  the  filter  and  fixed  lag  smoother.  These  robust  methods  have  been 
applied  to  simulated  and  real  tracking  data  to  obtain  Improved  estimation 
performance  In  the  presence  of  wild  observations. 

INTRODUCTION 

Robust  filtering  and  smoothing  are  a  natural  extension  of  the  robust 
M-estlmates  of  regression  developed  by  Huber  [1].  The  robust  M-estlmatas 
provide  a  natural  treatment  of  outlying  observations  and  have  been  ex¬ 
tremely  successful  In  dealing  with  outliers  In  other  data  reduction  pro¬ 
blems  .[2]  and  [3].  The  extension  of  the  M-estlmate  methods  provides  a 
fresh  approach  to  the  problems  caused  by  outliers  In  filtering  and  smooth¬ 
ing  applications.  Robust  methods  for  estimation  are  designed  to  perform 
well  when  observations  from  contaminating  distributions  art  present.  The 
conventional  estimation  techniques  of  least  squares,  maximum  likelihood, 
minimum  variance,  etc.  may  become  useless  when  the  observations  are  con¬ 
taminated  by  gross  outliers  or  wild  data  points.  When  using  these  esti¬ 
mation  methods,  outliers  are  often  treated  by  testing  the  residuals.  If 
It  Is  decided  that  a  residual  Is  statistically  too  large,  the  correspond¬ 
ing  observation  Is  declared  an  outlier  and  Is  not  processed.  These  hypo¬ 
thesis  testing  methods  are  ofterj  successful  If  only  a  small  number  of  out¬ 
liers  are  present  but  breakdown  for  larger  proportions  of  outlying  obser¬ 
vations.  Also,  In  order  for  outliers  to  be  detected,  they  must  be  re¬ 
latively  large  compared  to  the  measurement  noise.  The  detection  methods 
based  on  testing  of  residuals  are  relatively  Insensitive  to  small  outliers 
which  leads  to  an  Inflation  of  the  mean  square  estimation  error.  Thus, 
methods  for  treating  outliers  should  be  evaluated  on  their  ability  to 
achieve  a  small  mean  square  estimation  error  as  well  as  their  ability  to 
offer  protection  from  gross  outliers. 

Very  little  development 'has  appeared  on  the  application  of  robust 
estimation  techniques  to  filtering  and  smoothing.  The  most  notable  work 
In  this  direction  Is  that  of  Masrallez  and  Martin  [4].  Their  development 
of  the  application  of  M-estlmates  to  the  Kalman  filter  Is  mainly  theoret¬ 
ical.  The  emphasis  here  Is  on  the  development  of  some  practical  results  on 
the  application  of  M-estlmates  to  filtering  and  smoothing.  We  have  ap¬ 
plied  these  mathods  to  filtering  and  smoothing  of  tracking  data  from  tra¬ 
jectory  measurement  systems  at  WSMR,  Using  simulated  tracking  data  we 
have  also  performed  extensive  Monte  Carlo  evaluation  of  filtering  methods 
based  on  M-estlmates  to  determine  the  conditions  for  whleh  we  can  expect 
to  benefit  from  the  application  of  these  methods. 
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m-estimates  for  regression 


Given  scalar  observations  y^,  1  ■  1,  N  of  a  linear  model 

yt  -  XjQ  +  e1  (1) 

where  Xj  Is  a  row  vector  of  known  Independent  variables  and  Is  a  random 

error  term  we  want  to  estimate  the  unknown  p-vector,  e.  The  M-estlmate 
of  e  minimizes. 

» fa  -  v>'«)  •  ■  « 

where  «(•)  Is  a  specified  function  and  s  Is  a  robust  measure  of  dispersion 
of  the  residuals,  -  X^e.  Minimizing  (2)  by  differentiating  with  respect 

to  e  gives . 

N 

*  ♦  ( (y1  -  xiej/s)  -  0  (3) 

where  Is  the  derivative  of  p  and  9  Is  tlje  M-estIraate  of  9.  .(3)  Is  the 
analog  of  the  normal  equations  In  least  squares  estimation.  9  Is  computed 
Iteratively  by  applying  a  weighted  least  squares  algorithm  to  (3).  For 
details  see  [2]. 

• 

Rather  than  specifying  the  function  p,  M-estlmates  are  usually  de¬ 
scribed  by  specifying  the  function  Several  i|i  functions  have  been  pro¬ 
posed  In  the  literature.  The  only  functions  considered  here  are  varia¬ 

tions  of  the  one  proposed  by  Hampel  [S3.  The  Hampel  ♦  function  with  break¬ 
points  a,  b,  c,  denoted  by  Ha(a,b,c)  Is  glvep  by 


The  M-estlmates  can  also  be  applied  to  regression  problems  having  vector 
observations  and  to  nonlinear  regression  problems.  If  the  probability 
density  function  p  of  the  observetlons  Is  releted  to  «|>  by  pVp  *  -♦»  the 
resulting  M-estlmate  Is  maximum  likelihood.  For  any  tji  function  we  call 

*’pa  pseudo-density  and  derive  filters  In  some  convention!!  weys  with  the 
density  function  of  the  observations  replaced  with  a  pseudo-density. 

APPROPRIATE  NON- GAUSS I AN  FILTERING 

Assume  that  theStitpx(k)  of  the  process  being  observed  Is  governed 
by  the  discrete  linear  model, 

x(k+l)  »  e(K+l,k)  x(k)  +  u(k)  (5) 
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where  the  stats  vector  x(k)  Is  an  n-vector,  u(k)  Is  a  Gaussian  state  noise 
vector  with  zero  mean  and  covariance  QCk).  e(.k+1,k)  Is  an  n  x  n  state 
transition  matrix.  Scalar  observations*  z(k),  of  the  process  are  given  by 

z(k)  ■  H(k)x(k)  +  v(k)  (6) 

where  H(k)  Is  a  row  vector  and  v(k)  Is  a  measurement  noise  error  which  may 
be  contasilnated  by  outliers. 


In  order  to  derive  robust  filters  corresponding  to  the  M-estimates  of 
regression,  we  use  the  results  of  Masrellez  [6]  on  approximate  non-Gausslan 

filtering.  Masrellez  obtains  an  approximate  condltloral  moan  of  p(x(k)JZk) 

where  Zk  Is  the  collection  of  observations,  Z**  ■  (z(l),  z(2),  — ,  z(k)>. 

Using  Bayes  rule  p(x(k)|Zlc)  Is  given  by 

p(x(k)|Z*c)  ■  ofzfkllxtkDpfxflOJZ*"*)  (7) 

p(z(k)J2k-1) 

In  order ,,to  derive  a  useful  approximation  to  the  conditional  mean, 
E[x(k)|ZK],  of  (7),  Masrellez  assumes  that  p(x(k)|Z*“  )  Is  Gaussian  with 

mean  x(k[k-1)jmd  covariance  P(k1|k'-1).  The  resulting  approximate  condi¬ 
tional  mean,  x(kj k)  is  given  by 

x(k | k)  •  x(k|k-1)  +  P(k|k-l)HT(k)g(z(k)),  (8) 

where  g(z(k}  Is  the  scalar 

9(z(k))  - -p’(z'(k)|Zk-1)?j^jii(t(k)|Zl<-1)  (9) 

M*«r«11,z  .Iso  d.r1v«*  th.  iKond  mount,  P(k|k),  of  p(x(k)|Zk).  Tho  ro- 
suit  Is. 

P(k|k)  ■  P(k|k-1)  -  P(k|k-l)HT(k)G(z(k))H(k)P(k|k-l),  (10) 

with 

a(z(k))  •  (ID 

A  second  method  for  approximate  non-Gausslan  filtering  is  the.margl- 
nal  maximum  likelihood  filter.  In  this  case  we  find  the  estimate  x(k|k) 

which  maximizes  (7).  In  this  derivation  we  also  assume  that  p(x(k) jZ^"1) 

* 

Is  Gaussian.  .The  resulting  equation  for  x(k|k)  Is 

x(k|k)  ■  x(k|k-l)  -  P(xlk-1  )HT(k)p\j:lk)  jx( k  1  k) (u) 

The  estimate  given  by  (12)  Is  of  the  same  form  as  the  conditional  mean  estl- 
nate  given  by  (8).  The  difference  in  the  two  estimates  is  that  the  right 
hand  side  of  (12)  depends  on  x(kik)  while  the  right  hand  side  of  (8)  depends 
)n1y  on  the  predicted  estimate,  x(k|k-1).  Thus,  (12)  requires  Iteration  to 

)bta1n  the  estimate  and  If  In  the  first  Iteration  of  (12),  we  substitute 

x(ktk-l)  for  x(kjk)  on  the  right  hand  side,  the  resulting  estimate  on  the 
first  iteration  will  be  Identical  to  the  conditional  mean  estimate  of  (8). 
Thus,  the  maximum  likelihood  filter  may  be  regarded  as  a  correction  to  the 
conditional  mean  filter. 
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.  ROBUST  FILTER  EQUATIONS 

Th«  robust  filtering  aquations  corresponding  to  M-estlmatas  are  obtain¬ 
ed  by  rtplaclnfl  tha  danslty  function*  p(z(k)|Zk  )»  In  (9)  or  (12)  by  * 


pseudo-density*  e**,  where  p*  -  4-  ~  Is  »  daslrad  In- 

fluanca  function.  This  substitution  results  In  tha  following  aquations  for 
tha  conditional  naan  robust  flltar. 

„/ L.  I'  L.  1  \uT  / 1_  \  m  Uflflv/lt  I  1  \  ll.t 


idt|k)  -  x(k|k-D  ♦  w 

k  k 

,{k|k)  -Kklk-D  -/.'(^y»|<iy(k|k-1)HT(lt)H(k)ftk|k-l).(1«) 

<hara  *'  Is  tha  derivative  of  a  and  r(k)  ■  x(k)  -  H(k)x(k|k-1).  The  flltar 
aquations  arecompletad  by  tha  usual  Kalman  filter  equations  for  tha  pre¬ 
dicted  moments. 

In  order  to  Insure  tha  robustness  of  tha  filter  described  by  (13)  and 
(14),  the  dispersion  sk  of  tha  predicted  residuals  must  be  specified  so 

that  It  Is  Insensitive  to  outliers.  We  used  the  MAD  estimate  of  sfc  com¬ 
puted  from  past  residuals  as 

tk  ■  median  I  z(k-J)  -  HT(k-j)x(k-4|k«J-1)  I  /.674S  (IS) 

k  j-O.N-l  I 

whore  N  Is  a  suitably  chosen  Integer. 

•  The  robust  maximum  likelihood  filter  Is  obtained  bv  replacing  tha  den¬ 
sity  p(z(k)|Zk“1)  In  (12)  by  a  pseudo-density.  The  resulting  filter  Is 
given  by  _  ..  *  . 

s»iu  •  !(k|k-i>  -  ,  (m  -  0«) 

In  (16)  we  use  x(kjk-l)  to  denote  the  mean  of  p('x(k) jZ^"1 ).  We  use  (14)  to 

gRSiWtS'iaas  asssrtrW-  iSraiiSfi?  ssvs 

i(kilj  •  xuik-D  -  nmj-uiftu.  *  (»w  -  wwifelu) .  o»> 


starting  with  x(k|k)  •  x(k|k-1)  so  that  tha  first  estimate  x(klk)  Is  the 
conditional  mean  estimate,  x(kjk). 

EVALUATION  OF  THE  ROBUST  FILTER 

Evaluation  of  tha  robust  filtering  methods  described  above  » was  dona 
with  a  view  toward  eventual  application  to  trajectory 
■nphasls  In  the  evaluation  was  on  simulated  rather 
data.  This  allows  a  quantitative  determination  of  any  advantages  In  the 
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u$«  of  robust  filtering  in  the  presence  or  outliers  ano  a  iso  any  toss  in 
efficiency  using  robust  methods  whan  no  outliers  are  present.  The  simu¬ 
lated  trajectory  Is  that  of  a  constant  velocity,  level  flying  aircraft. 

The  measurements  are  of  position  In  each  of  threw  cartesian  components 
with  added  measurement  noise  which  Is  contaminated  by  outliers.  The  filt¬ 
er  model  assumes  the  trajectory  to  have  constant  acceleration  In  each  co¬ 
ordinate.  The  filter  for  each  coordinate  has  a  small  acceleration  state 
noise,  Q(k)  *  S.  The  outlier  contamination  Is  controlled  by  a  two  state 
Markov  chain  with  a  transition  possible  at  each  measurement  time.  An  out¬ 
lier  is  added  to  the  measurement  If  the  Markov  chain  is  In  state  two  and 
no  outlier  is  added  If  the  chain  Is  In  state  one.  The  transition  prob¬ 
abilities,  Plj,  are  used  to  determine  the  percentage  of  outliers  contam¬ 
inating  the  measurements  and  also  the  length  of  runs  of  outliers  in  the 
measurements.  The  magnitude  of  the  outlier  contamination  Is  C  •  fR»  1.a.» 
a  constant  multiple  of  the  measurement  noise  standard  deviation. 

Using  the  simulated  trajectory  data  a  Monte  Carlo  evaluation  of  ro¬ 
bust  filtering  was  performed.  The  rms  estimation  error  was  computed  point 
wise  for  position,  velocity,  and  acceleration  using  a  sample  size  of 
twenty-five.  The  plots  of  the  rms  errors  for  each  of  the  conditions  test¬ 
ed  requires  far  too  much  space  to  present  here.  Instead,  thesa  results 
ere  summarized  by  time  averages  of  the  rms  error  in  position  and  velocity 
for  each  of  the  conditions  tested. 

Figure.  1  compares  the  average  rms  position  error  for  two  filters  us¬ 
ing  the  Hampel  <ji  functions  Ha{2,  3,  4)  and  Ha  (4,  4,  4).  Figure  2  gives 
the  rms  velocity  error  comparison  for  the  same  two  filters.  Also  indi¬ 
cated  In  Figures  1  and  2  are  the  Ideal  rms  error  values  which  were  obtain¬ 
ed  with  an  ordinary  Kalman  filter  with  no  outliers  present  and  using  a 
known ,mf as urement  covariance,  ■  400. A  The  Monte  Carlo  evaluation  of 

figures  1  and  2  was  made  with  a  measurement  noise  standard  deviation  of 
ifljT ■  20  ft.  Rk  was  unknown  to  the  filter, 

We  note  from  figures  1  and  2  that  neither  of  the  robust  filters  lose  much 
efficiency  from  the  Ideal  values  when  no  outliers  are  present.  The  er¬ 
ror  curves  In  figures  1  end  2  behave  as  expected.  Since  outliers  small  In 
relation  to  the  measurement  noise  are  hardest  to  detect,  the  error  curve 


We  see  from  figures  1  and  2  that  Ha  (2,  3,  4)  has  a  significantly  smaller 
mean  square  error  than  Ha  (4,  4,  4).  Except  for  the  way  In  which  the  dis¬ 
persion  of  the  residuals  Is  measured,  l.e.,  the  MAD  estimate  In  (15), 

Ha  (4,  4,  4)  Is  a  conventional  way  of  handling  outliers  In  a  Kalman  filter¬ 
ing  application.  Using  Ha  (4,  4,  4)  any  observation  whose  predicted 
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residual  Is  greeter  than  4  •  t  Is  not  processed  and  any  observations  whoso 
predicted  residual  Is  loss  than  4  •  s  Is  processod  as  an  ordinary  Kalman 
filter  observation.  The  above  Monte  Carlo  evaluation  was  made  with  Martov 
chain  probabilities  Pax  ■  *05  and  P^  "  .5  which  gives  an  outlier  probabl- 
llty  of. 088  and. an  average  outlier  run  length  of  three.  In  order  to  re¬ 
duce  the  average  rms  errors,  we  pull  In  the  breakpoints  of  the  Hampel  * 
function.  Figures  3  and  4  compare  the  average  rms  errors  in  position  «nd 
velocity  for  the  Ha  (1,  2,  3)  and  Ha  (2,  3,  4). 
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Figures  3  and  4  were  made  using  the  same  outlier  proportions  and  measure¬ 
ment  noise  standard  deviations  as  figures  1  and  2. 


The  iterated  filter,  1 . e. »  the  approximate  maximum  likelihood  filter 
given  by  (14),  (16).  and  (17)  was  also  evaluated  under  the  same  conditions 
as  the  conditional  mean  filter.  Comparison  of  the  average  mean  square  er¬ 
rors  for  the  conditional  mean  and  maximum  likelihood  filters  showed  no 
discernible  differences. 


ROBUST  SMOOTHING 

A  simplified  robust,  fixed  lag  smoother,  was  derived  in  a  similar 
manner  to  the  robust  conditional  mean  filter  derivation.  We  derive  an  ap¬ 
proximate  conditional  mean,  E[x(k)|2l(,fn]tOf  the  density  p(x(k)|Zl{4,n)  using 
the  same  methods  and  assumptions  used  by  Masrellez  and  used  In  the  deri¬ 
vation  of  the  robust  filter.  The  robust  smoothed  estimate  Is  given  by 

x(k|k+n)  .  J(k|k)+P<k|k-1)  *  >T(VJ 

'  >1  'k+J  \  *k+j  /  (18) 

In  obtaining  (18)  the  fixed  lag  smoother  has  been  greatly  simplified  by  leav¬ 
ing  out  the  state  noise  In  the  forward  Interval,  x(kjk)  In  (18)  Is  the  ro¬ 
bust  filtered  estimate  described  by  (13)  and  (14).  Is  a  robust  measure 

of  dispersion  of  the  residuals,  z(k+j)-H(k+j)x(k+j|k-1).  Several  possibi¬ 
lities  exist  for  computing  a  useful  dispersion  measure,  s^j.  The  slmpllst 

method  and  the  one  used  to  obtain  the  smoother  evaluation  given  below  Is  to 
make  s^j  a  constant  s^j  »  sk  and  then  compute  sfc  by  (15). 
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A  Manta  Carlo  evaluation  of  the  simplified  fixed  lag  smoother  was 
performed  using  the  same  simulated  trajectory  as  was  used  for  the  filter 
evaluation.  Measurement  noise  having  a  standard  deviation  of  SO  feet  was 
added  to  the  simulated  positions.  The  measurement  noise  standard  dev1a~ 
tlon  was  unknown  to  the  smoother.  The  forward  smoothing  Interval  had  a 
length  of  n  ■  20  which  represents  a  one  second  smoothing  time.  The  out* 
Her  proportions  and  run  lengths  were  the  same  as  for  the  filter  eval¬ 
uation.  A  sample  size  of  ten  was  used  for  the  Monte  Carlo  evaluation  of 
the  smoother. 


Figures  5  and  6  display  the  average  rms  position  and  velocity  esti¬ 
mation  errors  obtained  using  t}ie  robust,  fixed  lag  smoother  with  the 
Hampel  <|>  functions,  Ha  (2,  3,  4)  and  Ha  (4,  4,  4).  Also  noted  In  fig¬ 
ures  5, and  6  Is  the  Ideal  average  rms  values  which  were  obtained  using  the 
smoother  with  no  outliers  and  a  known  measurement  covariance,  Rk  ■  2500. 

The  robust  smoother  using  Ha  (4,  4,  4)  Is  representative,  except  for  the 
measurement' of  »k,  of  a  conventional  way  of  handling  outliers  in  an  opti¬ 
mal  smoothing  application.  Wa  note  that  either  of  the  smoothers  offers 
good  protection  from  vary  large  outliers  but  that  Ha  (2,  3,  4)  results  In 
a  significantly  smaller  estimation  error  when  small  outliers  are  present. 


ROBUST  FILTERING  USING  GAUSSIAN  MIXTURES 


Robust  estimation  methods  deweight  and/or  reject  outlying  observations 
by  specifying  observational  densities  or  pseudo-densities  having  long, 
heavy  tails.  We  can  also  realize  a  long,  heavy  tailed  density  by  using  a 
Gaussian  mixture.  Suppose  we  replace  p(z(k)|x(k))  In  (7)  or  (12}  by  the 
Gaussian  mixture  pseudo-density, 

p(z(k)|x(k))  ■  E  a1N(z(k)  -  H(k)x(k)  -  aj^,  Rk)  (19) 


where 

N(z(k)  -  H(k)x(k)  -  aj^,  RR) 


We  do  not  require  that  Ze^  ■ 

tered  at  Ij^and  each  ha  vino  s 
Inf’lnlta.  Tha  locations,  ak, 
Using  (19)  we  obtain  p(z(k)|Z 


1.  Thus,  we  have  Individual  Gausslans  cen- 

tandard  deviation  R^.  The  sum  In  (19)  may  be 

and  the  amplitudes,  a, ,  are  free  parameters. 

• 
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(21) 


pUOOlZ**1)  -  £  a^N^z(k)  -  H(k)xCk|k-l)-«^15  .H(k)PCkJlt-l)HTk}+RIcJ 
In  obtaining  (21)  we  have  again  assumed  that  p(x(k) JZ54"1 )  Is  Gaussian. 


Substituting  (21)  Into  (8)  and  (9)  with  the  variance  of  the  residuals, 
H(k)P(k  Ic-ljHlkJ+R^treplaced  with  an  estimated  value,  s^2,  gives 


In  (22)  Is  the  weighted  average 

v  ■  fi*k0- 

where  the  weights  are  given  by 

o.N^k)  -  H(k)x(k|k-1 )  -  a£i},  s,2) 
1  tojN^k)  -  H(k)x(k|k-1)  -  aW/,sk2) 


(22) 

(23j 

(24) 


The  filter' equations  (22)i  (23),  and  (24)  are  Identical  In  form  to  the 
simplified  pseudo-Bayes  filter  by  Ackerson  and  Fu  [7]  for  adaptive  filter¬ 
ing  when  the  mean  of  the  measurement  noise  Is  unknown.  Using  (10)  and  (11) 
the  conditional  covariance  Is 


P(kjk)  •  P(kjk-I)  -  P(k|k-1)H(k)(Y$  2  -  fr.k^* 5sf)wTk)P(k | k«l ) , (25) 


where 


(»k°  -  \)‘  ■  “((‘k11  ■  *k)‘ 

Although  the  above  sums  may  be  theoretically  Infinite,  we  only  need 
to  calculate  the  relatively  few  terms  In  the  sums  which  have  significant 
values  of  the  weights,  Wj.  Thus,  wo  compute  only  those  terms  In  the  sums 

for  which  |z(k)-N(k)x(k|k-l)-aj^[/5|<*  4.  With  this  simplification  the 
amount  of  extra  computation  required  to  Implement  this  robust  filter  Is 
relatively  small.  The  locations,  aj^,  produce  a  smooth  pseudo-density 
If  they  are  chosen  as  zero  and  odd  Integral  multiples  of  sk,  a^  ■  0, 

ak^  "  59r(i)(2l1|  -  l)sk.  |1|  -  1.  We  have  also  tested  the  filter  with 

»  1  •  k,  (1 |  *  0.  The  value  of  the  residual  dispersion,  s^,  Is 

still  computed  by  (15).  Several  different  choices  of  the  amplitudes  have 
been  tested.  The  most  extensive  testing  hat  been  done  with  ■  1  and 

«,  -  V  (H|  +  1). 
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Soma  robust  filters  using  the  Gaussian  mixture  formulation  were  also 
evaluated  via  Monte  Carlo  testing.  These  filters  were  tested  using  the 
same  simulated  trajectory  data  and  under  the  same  outlier  and  measurement 
noise  condl tlons  as- the  robust  filters  using  Hampel  functions.  The  sam¬ 
ple  size  for  Monte  Carlo  was  twenty-five*  the  Markov  transition  probabi¬ 
lities  were  Pji  ”  *05  and  *  <5  and  the  measurement  noise  standard 


deviation,  which  was  unknown  to  the  filter  and  estimated  by  (15) ,  was 
20  feet.  Figures  7  and  8  present  the  average  rms  position  and  velocity 

;ontaminated  by 
Iters  use^  In  gen- 

i.  «<  ■  yen  i  + 1). 


errors  for  a  Gaussian  mixture  filter  with  observations  contaminated  by 
various  magnitudes  of  outliers.  The  Gaussian  mixture  filters  usejs 

eratlng  Figures  7  and  8  used ;magn1 tudes  of  the  Gausslans t 

Two  different  Gaussian  mixture  filters  are  represented  In  Figures  7  and  8,' 
one  with  Gauss  Ians  at  all  Integral  multiples  of  s^  and  one  with  Gausslans 

at  zero  and  odd  Integral  multiples  of  sk.» 
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Each  Gaussian  component  of  P(z(k)|Z  •)  has  standard  deviation  s^.  There 

Is  very  little  difference  In  the  estimation  errors  obtained  for  the  two 
filters  of  Figures  7  and  8.  The  filter  with  Gausslans  at. only  the  odd 
multiples  of  s^  Is  computationally  less  complex.  Figures  9  and  10  give 

the  results  of  the  Monte  Carlo  evaluation  of  a  Gaussian  mixture  filter 


This  filter  appears  to  give  somewhat  smaller  errors  than  the  other  Gaus¬ 
sian  mixture  filters  and  also  slightly  smaller  errors  that  the  robust 
filter  which  uses  Ha(l,  2.  3).  We  note  that 'using  •  1  with  Gausslans 

at  all  Integral  multiples  of  does  not  result  in  a  useful  filter  since 

It  has  a  zero  Influence  function  and  therefore  does  not  produce  any  error 
correction.  43 


CONCLUSIONS 


Two  methods  band  on  M-estimates  have  bean  presented  for  robust  filt¬ 
ering  and  smoothing,  one  using  the  Hampel  function  with  various  break¬ 
points  and  the  other  which  models  the  observation  error  as  a  Gaussian  mix¬ 
ture.  These  robust  filtering  methods  were  subjected  to  a  Monte  Carlo 
evaluation  using  simulated  trajectory  data  from  an  aircraft  tracking  ap¬ 
plication.  The  results  of  this  evaluation  show  that  both  of  .these  robust 
filtering  methods  give  a  significant  reduction  In  average* ms  estimation 
error  for  small  outliers  compared  to  a  more  conventional  way  of  treating 
outliers  In  an  optimal  filtering  application.  The  tests  also  suggest  that 
the  Gaussian  mixture  robust  filter  methods  offer  the  most  promise  for  ap¬ 
plication  and  future  testing.  Further  evaluation  of  robust  filtering 
methods  under  more  severe  trajectory  applications  are  necessary. 
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COMPOUND  FREQUENCY  DISTRIBUTIONS 

A  METHOD  FOR  ESTIMATING  STATISTICAL 
PARAMETERS  FROM  AN  ADULTERATED  SAMPLE 

Donald  W.  Rankin 


Army  Materiel  Test  and  Evaluation  Directorate 
US  Army  White  Sands  Missile  Range 
White  Sands  Missile  Range,  New  Mexico  88002 


ABSTRACT .  When  a  sample  is  contaminated  by  extraneous  "outliers'’, 
computation  of  the  higher  statistical  moments  may  contain  large  errors.  The 
proposed  method  treats  these  "outliers"  as  members  of  another  "unwanted" 
population,  and  assumes  that  they  perturb  the  distribution  minimally  near 
the  maximum  ordinate  (mode). 

i  The  distribution  is  studied  only  near  this  maximum  ordinate.  A  simple 

l  curve  (a  parabola,  say)  is  fit  by  the  method  of  least  squares  and  the  various 

|  derivatives  are  evaluated  at  this  maximum  ordinate.  Not  only  the  usual  statis- 

j  tical  parameters  (mean,  variance),  but  also  the  proportional  number  of  "outliers" 

turn  out  to  be  expressible  as  simple  functions  of  these  derivatives. 

I.  THE  PROBLEM 

Statistical  analysis  usual ’y  requires  that  certain  a  priori  assumptions 
be  made;  e.g.,  a  certain  population  is  normally  distributed.  From  time  to  time, 
however,  a  test  will  reveal  that  a  sample  has  been  drawn  which  is  incompatible 
with  the  basic  assumptions. 

An  example  which  quickly  comes  to  mind  is  the  distribution  of  aerial 
bombing  scores.  For  many  samples,  the  assumption  of  a  normally  distributed 
population  appears  to  be  invalid  —  frequency  in  the  "tails"  is  far  too  high. 
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Before  abandoning  the  postulation  of  normality,  let  us  address  the 
problem  from  a  different  point  of  view.  Keeping  the  example  of  aerial 
bombing  scores,  suppose  that  in  a  sample  of,  say,  100  bonding  runs,  the 
bombardier  misidentified  the  intended  aiming  point  on  ten  occasions.  It 
is  obvious  that  only  90  scores  will  be  drawn  from  the  "correct"  population 
(i.e.,  the  population  for  which  we  have  postulated  normality),  while  the 
remaining  ten  will  come  from  populations  with  displaced  means.  To  choose 
terminology,  we  shall  say  that  the  sample  is  adulterated  by  the  ten  runs 
from  unwanted  sources. 

How  the  density  function  is  affected  by  mixing  different  populations 
is  seen  in  Figure  I.  The  lower  curve  is  simply  the  normal  curvo  in  which 
o  ■  1  and  N  ■  100.  Adding  adulteration  from  two  extraneous  populations 
(a  -  1,  m ■  -2,  N  ■  20,  and  a-  1,  y  ■  2,  N  ■  10)  yields  the  density  function 
for  the  compound  frequency  distribution,  illustrated  by  the  upper  curve. 

It  is  apparent  at  once  that  the  "tails"  are  abnormally  thick.  It  should 
hlso  be  noted  that  the  density  function  is  deformed  least  near  the  mean. 

If  the  mean  of  the  extraneous  bits  of  data  is  displaced  by  much  more 
than  2a  ,  the  effect  on  the  center  of  the  sample  distribution  is  virtually 
nil.  Can  the  parameters  of  the  desired  population  be  recovered  by  studying 
the  sample  distribution  only  near  its  center? 
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Differentiating,  we  find  that 


0'  (Z) 


.  -1'22’  «d 


011  (Z)  ■  — -i-i —  , 

/Tit 


-1/2  Z2 


Evaluating  these  expressions  at  the  naxinun  ordinate,  we  find  that 


Z  -  0 


0  CO) 


✓ST 


0 •  (0)  -  0 
0”  (0)  -  - 


Expressing  0  (Z)  in  series  form, 


0  CZ) 


i r  z*  z4 

/5F  [_  '  T  T- 


7®  7  ® 

“  75“  +  357“  " 


It  is  easy  to  see  that  ior  small  values  of  Z  —  say  | Z | <0.4  —  the  first  two  terms 
form  a  sufficiently  good  approximation.  Note  that  for 


0  (Z)  .  — L_  Ti  -  -|i  1  and  Z  -  0, 

/3tt  L  J 


0'  (Z)  -  - 


0"  (Z) 


0  (0) 


✓ET 


0*  (0)  -  0 


0"  (0) 


/ET- 


ln  other  words,  at  the  point  of  the  maximum  ordinate  (Z  -  0),  the  approximating 
parabola  and  its  pertinent  derivative*  take  on  exactly  the  aama  values  ea  the 
probability  density  function. 


Since  0  (Z)  is  a  probability  function, 


0  (Z)  dZ  -  1. 


In  an  actual  case,  a  sample  of  size  N  will  be  drawn,  consisting  of  N  values 


of  the  form 


Bach  y^  can  be  thought  of  as  a  deviation  about  a  certain  origin;  i.e.,  as 
an  abscissa.  Since  each  y^  occurs  with  frequency  1,  we  have  immediately 

N 

fCy^  ■  1  and  l  f(yi)  *  N. 
i»l 

Let  us  make  two  assumptions  ->  first  that  the  desired  population  (call  it  the 
"Z  population")  is  normally  distributed  —  second  that  the  extraneous  members  of  the 
sample  are  clustered  about  points  far  enough  removed  from  the  mean  of  the  Z  popu¬ 
lation  that  the  frequency  distribution  is  minimally  perturbed  near  the  center. 

Let  Nz  denote  the  number  of  members  of  the  sample  which  are  drawn  from  the 
desired  population.  Let  y  and  o  be,  respectively,  the  mean  and  standard  deviation 


of  the  y^'s  of  this  Z  population.  Then 


. . . 


,7,7..  ^  .  i  ' 


:::  : r 


Z  ■  y  "  H  is  the  standardised  variable, 
o 

allowing  our  previously  developed  expressions  in  Z  to  hold.  Note  that 

00  Q0 

/  0  (Z)  dZ  -  J  -i-  0  (-£**•)  dy  -  1. 

kaOO  '  * 

If  we  could  identify  the  members  of  the  Z  population,  we  could 
determine 

n2  ■  l  fCy4) 

merely  by  counting.  This  cannot  be  done.  But  we  note  that 


N. 


0  dy  "  n2 


Of  -  \0  /  "Z 

choosing  to  regard  y  (without  a  subscript)  as  a  continuous  variable,  and  thus 
considering  f(y)  as  an  ordinate  rather  than  as  a  frequency,  we  find. 


N. 


f(y)  • 

f 1  (y) 


(z) 


0 

-N  (y-p) 


N  _  (y-u)2 
•  -  e  2cr 


o»^i” 
-N 


[‘  -  M]  •  •  HP 
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Evaluating  thaaa  at  tha  maximum  ordinata  (y  ■  ji) ,  ve  find  that 


f  (?) 


V2 r 


V  (y) 


f"  (?) 


-N, 


Solving  for  N(  and  or  yialda  two  fundamantal  axpraoaiona. 


a 

er  - 


--LJ2L 


t"  (?) 


27T  fJ(y> 


f"  (?) 


For  actual  computation,  tha  form 


«r  f  (?)  -^2  rr 


may  be  praf arable 
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It  raaalna  to  avaluata  tha  iud  (y  la  tha  baat  aatiaata  of  W)  and  tha  valuau 
of  tha  function  and  its  aacond  derivative  at  that  point.  For  thla  va  uaa 
tha  approximating  parabola  (laaat  equerea  fit  *), 


f(  7)  -  K0  + 

f  *  < y>  -  + 

fM(y)  -  2k2 

Kiy  +  K2y2 

2K2  y 

Slnca 

f'cy)  ■  o, 

_  -I, 

V  -  .  1 

2Kj 

and 

Q 

1 

,* 

1 

i 

»i2 

of  couraa 

U 

f"C7)  -  2K2  . 

4K2  * 

Tha  raadar  la  raalndad  that  tha  fora  of  tha  dlatrlbutlon  la  aalactad 
from  crltarla  othar  than  tha  appaaranca  of  tha  raw  data.  For  example, 
a  dlatrlbutlon  which  followa  (Appendix  A»  Tabla  1.  Figure  II)  appaari  to 
ba  tri-modal,  dua  to  tha  praaanca  of  many  extrarieoue  outllete. 


*8ac  Appandlx  A 
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AfPRiMK  A 
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Tha  application  of  tha  not hod  of  looat  ■ quarto  la  greatly  foollitattd 
by  trout forming  tho  indapondant  variable  to  that  ito  tronifonu  hot  o  noon 
of  0  tad  on  incraaont  of  1.  By  way  of  llluotrotion,  In  Toblo  1,  y  lo  tho 
Indapondant  variable,  f  lto  froquoney  and  x  ita  trantfom. 


x  -  y  ■  34 
3 


o-y  -  3  <rx 


In  torma  of  tho  tronoforaod  varlablo  x,  a  Inca 
(n  la  any  poalelvo  lntogor) , 


t-i  -■  • 


g  (x)  ■  A0  +  Ax  X  +  Aj 


If 

Zx2 

Ix2f 

I*4 

II 

I*2 

Zx2 

lx4 

Z*2 


Z  i  z  f 

*2  .  ,W  £jgt 

Z  i  Z  x2 

Z*2  I*4 

Table  2  contalno  nuabora  uaoful  in  coaputation. 
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TABLE  2 


SUMMED  SQUARES  AND  FOURTH  POWERS  WITH  OTHER  USEFUL  NUMBERS 


i 

i 

i 

i 

II 

-i 

1*1 
-1  1 

*D 

2 

5 

10 

34 

70 

3 

7 

28 

196 

588 

4 

9 

60 

708 

2772 

5 

11 

110 

1958 

9438 

6 

13 

182 

4550 

26026 

7 

15 

280 

9352 

61880 

8 

17 

408 

17544 

131784 

9 

19 

570 

30666 

257754 

10 

21 

770 

50666 

471086 

11 

23 

1012 

79948 

814660 

12 

25 

1300 

121420 

1345500 

II  **  I  **♦ 


7t  -  3.141  592  654 
2*  =  6.283  185  307 


Suppose  it  is  desired  to  fit  •  ptrsbole  to  the  9  central  points  (Table  1) 


4, 

-  20,  2xf 

-  -2,  Zx2f  -  70 

Aq  - 

120  60) 

1 70  7081 

-  9960  -  3.593  074 

2772 

2772 

■ 

H 

< 

“2/60  - 

-0.033  333 

At  " 

« 

)  9  20 1 

160  701 

-  -570  -  -0.205  628 

2772 

2772 

X  ■ 

“A1 

-  -0.081  053 

2A2 

*<  l 

s 

23.756  84 

*  (X) 

.  v  -  “2 

_  -  3.594  424 

4*j 

g"  <x) 

-  -0.411  255 

• 

\(  S.  (x) 

-  2.95637 

or  ■ 

X 

f  g”(x) 

'y  ’ 

8.86911 

N  - 


r  ,<x)  V2 IT  -  26.64  or  27 
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THE  ERROR  INDUCED  BY  ASSUMING  A  PARABOLA  TO  APPROXIMATE  A  NORMAL  CURVE 

With  no  prior  knowledge  of  the  value  of  <r,  it  usually  happens  that 

rather  large  values  of  v  are  used  to  fit  a  parabola  (1.38  in 

er 

illustration  in  Appendix  A).  When  this  happens,  the  computed  values  of  N* 

and  <r  will  be  too  large.  9. 

,  “ZZ/2 

Table  3  shows  the  error  induced  by  the  approximation  1  -_Z_  ■  e 

2 

The  tabular  values  give  the  error  ait  a  proportion  of  g(x)»  the  maximum 
ordinate  (o~  ■  1) . 

If  the  mean  and  variance  can  be  estimated  (and  the  first  parabola  fit 
will  provide  a  rough  estimate) ,  the  error  at  each  value  of  the  independent 
variable  can  be  computed.  If  these  error  terms  are  subtracted  from  the 
corresponding  frequencies,  a  parabola  fit  to  these  "corrected  frequencies" 
will  exactly  reproduce  the  desired  parameters  with  no  residual  error 
(except  that  induced  by  errors  in  the  estimation  of  and  o) . 

Although  the  correction  function  is  exact,  it  should  be  remembered 
that  on  the  normal  curve  there  are  inflection  points  at  +  <r,  outside  of 
which  the  parabola  and  normal  curve  diverge  very  rapidly.  This  makes 
corrections  computed  for  points  outside  cr  dependent  upon  accurate  estimates 
of  and  <r.  For  example,  a  5X  error  in  the  estimate  of  (Twill  result  in 
a  correction  error  at  0.5  <r  of  only  0.3Z  of  the  maximum  ordinate,  but 
3.21  at  1.2  cr  —  a  tenfold  increase. 

The  histogram  of  the  data  from  Table  1,  with  the  best-fitting  normal 
curve  and  associated  parabola  is  illustrated  in  Figure  II. 
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TABLE  3 


J.  Z2 

e  2  Z 


1  -  ±  Z2 


z 

A  * 

1  22 

1  '  1 

Diff 

0.00 

1.0000 

1.000 

.00000 

0.02 

.99980 

.9998 

.00000 

0.04 

.99920 

.9992 

.00000 

0.06 

.99820 

.9982 

.00000 

0.08 

.99681 

.9968 

.00001 

0.10 

.99501 

.9950 

.00001 

0.12 

.99283 

.9928 

.00003 

0.14 

.99025 

.9902 

.00005 

0.16 

.98728 

.9872 

.00008 

0.18 

.98393 

.9838 

.00013 

0.20 

.98020 

.9800 

.00020 

0.22 

.97609 

.9750 

.00029 

0.24 

.97161 

.9712 

.00041 

0.26 

.96676 

.9662 

.00056 

0.28 

.96156 

.9608 

.00076 

0.30 

.95600 

.9550 

.00100 

0.32 

.95009 

.9488 

.00129 

0.34 

,94384 

.9422 

.00164 

0.36 

.93725 

.9352 

.00205 

0.38 

.93034 

.9278 

.00254 

0.40 

.92312 

.9200 

.00312 

0.42 

.91558 

.9118 

.00378 

0.44 

.90774 

.9032 

.00454 

0.46 

.89960 

.8942 

.00540 

0.48 

.89119 

.8848 

.00639 

0.50 

.88250 

.8750 

.00750 

0.52 

.87354 

.8648 

.00874 

0.54 

.86433 

.8542 

.01013 

0.56 

.85488 

.8432 

.01168 

0.58 

.84518 

.8318 

.01338 

0.60 

.83527 

.8200 

.01527 

0.62 

.82514 

.8078 

.01734 

0.64 

.81481 

.7952 

.01961 

0.66 

.80429 

.7822 

.02209 

0.68 

.79358 

.7688 

.02478 

0.70 

.78270 

.7550 

.02770 

Z 

.4** 

>-r 

Diff 

0.70 

.78270 

.7550 

.02770 

0.72 

.77167 

.7408 

.03087 

0.74 

.76048 

.7262 

.03428 

0.76 

.74916 

.7112 

.03796 

0.78 

.73771 

.6958 

.04191 

0.80 

.72615 

.6800 

.04615 

0.82 

.71446 

.6638 

.05068 

0.84 

.70272 

.6472 

.05552 

0.86 

.69087 

.6302 

.06067 

0.88 

.67896 

.6128 

.06616 

0.90 

.66698 

.5950 

.07198 

0.92 

.65495 

.5768 

.07815 

0.94 

.64288 

.5582 

.08468 

0.96 

63078 

.5392 

.09158 

0.98 

.61866 

.5198 

.09886 

1.00 

.60653 

.5000 

.10653 

1.02 

.59440 

.4798 

.11460 

1.04 

. 5B228 

.4592 

.12308 

1.06 

.57018 

.4382 

.13198 

1.08 

.55811 

.4168 

.14131 

1.10 

.54608 

.3950 

.15108 

1.12 

.53409 

.3728 

.16129 

1.14 

.52215 

.3502 

.17195 

1.16 

.51028 

.3272 

.18308 

1.18 

.49848 

.3038 

.19468 

1.20 

.48675 

.2800 

.20675 

1.22 

.47511 

.2558 

.21931 

1.24 

.46357 

.2312 

.23237 

1.26 

.45212 

.2062 

.24592 

1.28 

.44078 

.1808 

.25998 

1.30 

.42956 

.1550 

.27456 

1.32 

.41845 

,1288 

.28965 

1.34 

.40747 

.1022 

.30527 

1.36 

.39661 

.0752 

.32141 

1.38 

.38589 

.0478 

.33809 

1.40 

.37531 

.0200 

.35531 

I  I  4  hi  «£■.»  *.  i/ii  V-'I'H  1  J.  l.lVk» 
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Interpolation  in  Table  3  can  be  avoided  by  choosing  values  of  cr  and 


jx  auch  that 


and 


0.02  X 


0.02  R» 


f 


(R  and  R*  ara  intagare). 

This  la  a  reasonable  procedure;  ainee  the  first  satinets  of  <r% 

probably  is  too  high,  it  is  lagltlaate  to  ehooaa  that  lower  value  which 
serves  the  purpose.  Table  4  is  presented  for  convenience  in  choosing 

■t 

Application  of  the  correction  technique  to  the  euaaple  of  Appendix  A 
is  shown  in  Table  5  and  following. 


TABLE  4 


VALUES 

1 

a 

.02 

.04 

.06 

.08 

.10 

.12 

.14 

.16 

.18 

.20 

.22 

.24 

.26 

.28 

.30 


OF  ox  WHICH  YIELD  CLASS  INTERVALS  OF  .02R 


1 

0 

a 

o 

SO 

.32 

3.125 

25 

.34 

2.94118 

16.66667 

.36 

2.77778 

12.5 

.38 

2.63158 

10 

.40 

2.5 

8.33333 

.42 

2.38095 

7.14286 

.44 

2.27273 

6.25 

.46 

2.17391 

5.55556 

.48 

2.08333 

5 

.50 

2 

4.54545 

.52 

1.92308 

4.16667 

.54 

1.85185 

3.84615 

.56 

1.78571 

3.57143 

.58 

1.72414 

3.33333 

.60 

1.66667 

1 


TABLE  S 


CORRECTION  OP  A  HYPOTHETICAL  FREQUENCY  DISTRIBUTION 


y 

f(y) 

X 

X  -  X 

°x 

Factor 

12 

0 

-4 

-1.42 

IS 

3 

-3 

-1.06 

.13198 

18 

2 

-2 

o 

i 

.02770 

21 

4 

-1 

-.34 

.00164 

24 

2 

0 

.02 

.00000 

27 

5 

1 

.38 

.00254 

30 

2 

2 

.74 

.03428 

33 

2 

3 

1.10 

.  5108 

36 

0 

4 

1.46 

l 

20 

0 

From  Appendix  A,  we  have 

x  -  -0.081  053 

g(x)  -  3.594424 

ox  -  2.95637 

l/ox  ■  0.338 

for  correction,  set  l/ox  ■  0.36 


Af 

£+Af 

x(£+A£) 

x2(f+Af) 

-0.47 

2.53 

-7.59 

22.77 

-0.10 

1.90 

-3.80 

7.60 

-0.01 

3.99 

-3.99 

3.99 

-0.00 

2. 

-0.01 

4.99 

4.99 

4.99 

-0.12 

1.88 

3.76 

7.52 

-0.54 

1.46 

4.38 

1 

13.14 

1 

I*** 

-1.25 

18.75 

-£•.25 

60.01 

.tf'iY 

r. 


$ 


to- 

I 


It  is  observed  that  for  x  ■  +  4,  the  entering  argument  lies  outside  the 
range  of  Table  3.  This  merely  means  that  the  original  curve  fit  spanned 
too  much  data.  The  values  for  x  ■  +  4  are  simply  dropped  from  further 
calculations. 

Fitting  a  parabola  to  the  "corrected"  frequencies,  we  find 


18.75  28 

60.01  196 


1994.72 


588 


588 


■l 


-2.25 


-0.08036 


28 


b2 


7 

28 


18 

60 


588 


.75  1 

.01  I 


-104.93 

588 


x 

yl 

a  00 


-0.22515 

23.32455 

3.40143 


*"(x)  -  -0.35690 


<r*  -  3.08713 


Oy  ■  9.26138 


3.39238 


-0.17845 
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The  consequence  here  of  dropping  the  dete  for  x  *  44  li  that  rarity, 
too  small  an  estimate  of  <r  .  Another  iteration  seems  in  order.  See 


Tables  6,  7  and  following 


TABLE  6 


DISTRIBUTION 


y 

f 

x  -  * 

Factor 

Ag 

X 

F  - 

x  F 

x2  F 

«■* 

f  +Ag 

15 

3 

-.88 

.06616 

-0.23 

-3 

2.77 

-8.31 

24.93 

18 

2 

-.56 

.01168 

-0.04 

-2 

1.96 

-3.92 

7.84 

21 

4 

-.24 

.00041 

0. 

-1 

4. 

-4. 

4. 

24 

2 

0.08 

.00001 

0. 

0 

2. 

0 

0 

27 

5 

0.40 

.00312 

-0.01 

1 

4.99 

4.93 

4.99 

30 

2 

0.72 

.03087 

-0.11 

2 

1.89 

3.78 

7.56 

33 

2 

1.04 

.12308 

-0.42 

3 

1.58 

4.74 

14.22 

2 

20 

-0.81 

19.19 

-2.72 

63.54 

19.19  28 

63.54  196 
588 


-  1982.12 
58B 


3.37095 


-2.72 

28 

7  19.191 
28  63.541 
588 


-0.09714 


■  -92.54 

588 


-0.15738 


•r-*?  u%  T‘-  -  ■ 


3?  - 

-0.30862 

y  ■ 

23.07413 

g(x) 

-  3.38594 

r”(x) 

-  -0.31476 

°x  - 

3.27981 

°y  “ 

9.83943 

N  - 

27.84  or  28 

1  - 

0.305  let 

1 

0.30 

x  -  -0.093, 

let 

TABLE  7 


FINAL  CORRECTION  OF  A  HYPOTHETICAL  FREQUENCY  DISTRIBUTION 


y 

f 

X  -  X 

«X 

Factor 

^R 

* 

X 

X0 

x2* 

15 

3 

-.80 

.04615 

-0.16 

2.84 

-3 

-8.52 

25.56 

18 

2 

-.50 

.00750 

-0.03 

1.97 

-2 

-3.94 

7.88 

21 

4 

• 

-.20 

.00020 

0 

4. 

-1 

-4. 

4 

24 

2 

.10 

.00001 

0 

2. 

0 

0 

0 

27 

5 

.40 

.00312 

-0.01 

4.99 

1 

4.99 

4.99 

30 

2 

.70 

.02770 

-0.09 

1.91 

2 

3.82 

7.64 

33 

2 

1.00 

. 10653 

-0.36 

1.64 

3 

4.92 

14.76 

I 

20 

-0.65 

19.35 

-2.73 

64.83 
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3.36286 


19.33  26 

»o  -  -1-64.63  196 

®i  -  -rl-U _  - 

28 

I  7  19.35 

d2  -  kjj  64»MX 

588 

x  -  -0.32378 

y  -  23.02267 

g(x)  ■  3. 37874 
g"(x)  -  -0.29929 
trx  -  3.35996 
<Ty  -  10.07988 
N  -  28.46  or  28 

Check t 

1  -  0.298 

tr 

x 

0.298  x  -  -0.097 


-0.09750 


-  -0.14964 
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APPENDIX  C 


EXAMPLES 


To  test  the  method, three  ■ ample#  of  size  150  were  drawn  from  Rand's 
Table*  of  Gaussian  deviates  and  were  adulterated  as  described. 

Problem  1 

Sample  starts  with  line  2206,  Page  45 
First  100  numbers  unbiased 
Next  35  numbers  biased  4-2.5 
Next  15  numbers  biased  -2.0 

Class  Interval  was  chosen  as  0.3,  yielding  the  sample  distribution  displayed 
in  Table  8,  For  the  population,  p  ■  0,  0*  ■  1,  The  "pure"  portion  of 

the  sample  (N  ■  100)  estimates  these  parameters  as  »  0.015,  cr  ■  0.943. 
The  total  sample  provides  the  useless  estimates  )i  ■  0.332,  <r  »  1.579 

The  curve-fitting  technique  provides  a  first  approximation  of  •  0.059, 

<r  ■  1.025,  N  -  109.  After  applying  the  corrections,  the  method  yields 
p  *0.062,  <r  “  0.920,  N  ■  99.  Computations  are  shown  in  Tables  9, 

10  and  following. 


*The  RAND  Corporation,  A  Million  Random  Digits  with  100,000  Normal  Deviates, 
Free  Press,  1955. 
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TABLE  8 


FREQUENCY  DISTRIBUTION,  PROBLEM  1 


y 

Mid-Point 

X 

Frequency 
Unbiased  Total 
Portion  Sample 

-4.5 

-15 

1 

-4.2 

-14 

0 

-3.9 

-13 

0 

-3.6 

-12 

1 

-3.3 

-11 

0 

1 

CM 

O 

-10 

2 

-2.7 

•9 

0 

-2.4 

-8 

2 

-2.1 

-7 

2 

3 

-1.8 

-6 

3 

4 

-1.5 

-5 

3 

7 

-1.2 

-4 

6 

8 

-0.9 

-3 

7 

8 

-0.6 

-2 

9 

9 

-0.3 

12 

12 

0.0 

0 

12 

12 

Frequency 


y 

Mid-Point 

X 

Unbiased 

Portion 

Total 

Sample 

0.3 

1 

14 

14 

0.6 

2 

10 

11 

0.9 

3 

8 

8 

1.2 

4 

8 

14 

1.5 

5 

3 

7 

1.8 

6 

0 

4 

2.1 

7 

2 

S 

2.4 

8 

1 

4 

2.7 

9 

5 

3.0 

10 

2 

3,3 

11 

2 

3.6 

12 

2 

3.9 

13 

2 

4.2 

14 

0 

4.5 

15 

0 

4.8 

16 

1 
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CURVE  PIT.  PROBLEM  1 


y 

f 

X 

x: 

-.9 

8 

-3 

— i 

-.6 

9 

-2 

-] 

-.3 

12 

-1 

-] 

0 

12 

0 

.3 

14 

1 

1 

.6 

11 

2 

2 

.9 

8 

3 

2 

Z 

74 

74  28 

"  1250  196  -  12.76190 

5B8 

A1  -  6/28  -  0.21429 

I  7  74  I 

a2  -  ±28  250  /  -  -0.54762 

588 

*  -  -A372A2  -  0.19565 

-  A0  -Aj/4A2  -  12.78287 
«x  -  3.41633 

To  compute  corrections 
1/<TX  -  .293,  Use  0.32 

0.32  x  -  .063,  Uss  0.06 

y  *  0.05870 

<5y  -  1.02490 


TABLE  10 

CORRECTED  CUKVC  FIT.  PROBLEM  1 


y 

f  x  -  x 

Factor 

At 

g 

X 

xg 

x*g 

*X 

-.9 

8  >1.02 

.11460 

-1.46 

6.54 

-3 

-19.62 

58.86 

-.6 

9  -  .70 

.02770 

-0.35 

8.65 

-2 

-17.30 

34.60 

-.3 

12  -  . 38 

.00234 

-0.03 

11.97 

-1 

-11.97 

11.97 

0. 

12  -  .06 

.00000 

0. 

12. 

0 

0 

0 

.3 

14  .26 

.00056 

-0.01 

13.99 

1 

13.99 

13.99 

.6 

11  .58 

.01338 

-0.17 

10.83 

2 

21.66 

43.32 

.9 

8  .90 

.07198 

-0.92 

7.08 

3 

21.24 

63.72 

Z 

74 

-2.94 

71.06 

8.00 

226.46 

B_ 

1  71.06  28 J 

-  1226.46  1961 

-  12 

.90286 

o 

J — its 

B1 

-  8.00/28  - 

0.28571 

• 

B, 

1 7  71.061 

-  1 28  226.461 

-0 

.68786 

588 

lx  ■  0.20768 

y  -  0.06231 
g(x)  -  12.93253 
Ox  »  3.06604 

<j^  -  0.91981 

N  -  99.4  or  99 
Check: 

1  -  0.326 

0.326  x  -  0.068 


PROBLEM  2 


Sample  stares  with  line  5622,  Page  113.  All  other  conditions  Identical  to 
Problem  1.  Results  are  shown  In  Table  11,  12  and  13  and  following.  The 

"pure"  portion  of  the  sample  yields  ■  0 ,  <r  ■  0.913.  The  total 

♦ 

sample  gives  y  ■  0.426,  <r  <■  1.728. 

The  curve-fitting  technique  provides  a  first  approximation  of  ■  -0.101, 
cr  -  1.011,  N  ■  112.  After  applying  the  corrections,  the  method  yields 

ji  -  -0.106,  cr  -  0.908,  N  -  102. 

PROBLEM  3 

Sample  starts  with  line  8371,  Page  168.  All  other  conditions  identical  to 
Problems  1  and  2.  The  "pure"  portion  of  tho  sample  yields  fi  ■  0.102, 
cr  ■  0.961.  The  total  sample  gives  p  *  0.386,  <r  »  1.508. 

The  curve -fit ting  technique  provides  a  first  approximation  of  y.  ■  0.123, 
<r  ■  1.230,  N  ■  136.  After  applying  the  corrections,  the  method  yields 
■  0.112,  T  *•  1.109,  N  ■  123,  Results  are  shown  in  Tables  14,  15  and 
16.  It  ehoul d  be  observed  that  this  particular  sample  contains  5  "bad" 
data  bits  in  the  interval  |x|  5  0.6,  more  than  8%  of  the  sample.  The 

method  cannot  identify  these  points,  with  the  result  that  the  computed 
values  of  N  and  cr  tend  to  be  too  large .although  they  are  still  better 
estimates  of  the  true  parameters  than  those  obtainable  from  the  entire 
sample. 
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TABLE  12 

» 

CURVE  FIT .  PROBLEM  2 


y 

Mid  Point 

X 

f 

xf 

x2f 

-0.9 

-3 

8 

-24 

72 

-0.6 

-2 

11 

-22 

44 

-0.3 

-1 

18 

-18 

18 

0. 

0 

12 

0 

0 

0.3 

1 

9 

9 

9 

0.6 

2 

10 

20 

40 

0.9 

3 

8 

24 

72 

£ 

76 

-11 

255 

76  28 

Ao  "  255  1961  -  13.19048 

588 

Ai  «  -11/28  -  -0.39286 

I  7  76| 

*2  “  128  2551  -  -0.58333 

588 

x  -  -0.33673 

fCx)  -  13.25662 

Ok  -  3.37088 

To  compute  corrections , 

1  -  0.297,  Use  0.32 
°x 

0.32T  -  -0.108,  Use  -0.10 

y  -  -0.10102 

<ry  -  1.01126 


N  -  112 


TABLE  13 


* 

t 

x  -  X. 

Factor 

Af 

8 

X 

xg 

x*g 

* 

-.9 

8 

rx 

-.86 

.06067 

-0.80 

7.20 

-3 

-21.60 

64.80 

-.6 

11 

-.54 

.01013 

-0.13 

10.87 

-2 

-21.74 

43.48 

-.3 

18 

-.22 

.00029 

0. 

18. 

-1 

-18. 

18. 

0. 

12 

.10 

.00001 

0. 

12. 

0 

0 

0 

.3 

9 

.42 

.00378 

-0.05 

8.95 

1 

8.95 

8.95 

.6 

10 

.74 

.03428 

-0.45 

9.55 

2 

19.10 

38.20 

.9 

8 

1.06 

.13198 

-1.75 

6.25 

3 

18.75 

56.25 

Z 

76 

-3.18 

72.82 

-14.54 

229.68 

72.82  28 

B  -  229.68  19.6  . 

0  588 

Bx  -  -14.54/28  - 

I  7  72.82) 

»  1 28  229.681  - 

2  588 

x  -  -0.35406 

y’  h  -0.10622 

g(x)  ■  13.42812 
■  3.02581 

<j^  ■  0.90774 

N  »  101.8  or  102 

Chock: 

1  -  0.330 

«k 

.330  x  -  -0.117 


-  13.33619 
-0.51929 
-0.73333 


,  ittBVBiiSi 


■-.mS 


Mid  Point 
-3.0 
-2.7 
-2.4 
-2.1 
-1.8 
-1.5 


TABLE  14 

FREQUENCY  DISTRIBUTION.  PROBLEM  3 


Frequency 
Unbiased  Total 

Portion  Samp la 


Mid  Point 
1.2 
1.5 
1.8 
2.1 
2.4 
2.7 


Frequency 
Unbiaaad  Total 

Portion  Sample 


-1.2 

-  4 

3 

6 

3.0 

10 

4 

-0.5 

-  3 

8 

9 

3.3 

11 

5 

-0.6 

-  2 

9 

11 

3.6 

12 

0 

-0.3 

-  1 

10 

11 

3.9 

13 

1 

0.P 

0 

13 

14 

4.2 

14 

0 

0.3 

1 

13 

13 

4.5 

15 

0 

0.6 

2 

12 

13 

4.8 

16 

2 

0.9 

3 

10 

10 

5.1 

17 

0 

TABLE  13, 


xf 

x*f 

Factor 

«x 

-27 

81 

-.96 

.09158 

••1.21 

-22 

44 

-.68 

.02478 

-0.33 

-11 

11 

-.40 

.00312 

-0.04 

0 

0 

-.H 

.00003 

0. 

13 

13 

.16 

.00008 

0. 

26 

52 

.44 

.00454 

-0.06 

30 

90 

.72 

.03087 

-0.41 

9 

291 

-2.05 

13.14286 


.  9/28  -  0.32143 


7  81 

28  291  *  -0.39286 

588 


x  «  0.40909 

f (x)  -  13.20860 

0^  «  4.10012 

y  -  0.12273 

Oy  -  1.23003 

K  -  136 

To  compute  &t,  uee 

1  -  0.28 

\ 

p/tr  *  0.12 


76 


XI 


78*95  28 

B-  -  1274 1 82  1961  -  13.23000 

588 

Bi  -  11.98/28  -  0,42786 

1  7  78.95  1 

B,  “  128  274.82  1  - -0.48786 

588 

0.43851 
f<x)  -  13.32381 

<rx  -  3.69533 
7  -  0.13155 

Oy  -  1 . 10860 

N  -  123 


Check: 


Is  the  result  effected  by  varying  the  span  of  data  used?  Let  us 
rework  Problem  3,  using  all  the  data  up  to  |y|  ■  1.8.  Even  though 

the  estimates  of  tr  and  p  are  poor,  iteration  of  the  method  quickly 
puts  data  for  |y| >  1.2  outside  the  range  of  the  correction  table,  and 
so  drops  them  from  the  calculations.  The  remaining  9  points,  when 
properly  corrected,  should  virtually  duplicate  the  results  earlier  obtained 
from  fitting  a  curve  to  ^  points.  The  results  compare  as  follows: 

9-point  fit:  p  ■  0.162,  or  ■  ,1.096,  N  ■  123 

7-pelnt  fit:  p  ■  0.132,  or  ■  1.109  N  *  123 


f 

f 
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TABLE  17 

13-POINT  CURVE  FIT.  PROBLEM  3 


y 

f 

X 

xf 

x*f 

Factor 

Af 

-1.8 

3 

-6 

-18 

108 

-1.36 

.32141 

•4.03 

-1.5 

5 

-5 

-25 

125 

-1.14 

.17195 

-1.16 

-1.2 

6 

-4 

-24 

96 

-.92 

.07815 

-0.98 

-0.9 

9 

-3 

-27 

81 

o 

1 

.02770 

-0.35 

-0.6 

11 

-2 

-22 

44 

-.48 

.00639 

-0.08 

-0.3 

11 

-1 

-11 

11 

-.26 

.00056 

-0.01 

0 

14 

0 

0 

0 

1 

o 

.00000 

0 

0.3 

13 

1 

13 

13 

.18 

.00013 

0 

0.6 

13 

2 

26 

52 

.40 

.00312 

-0.04 

0.9 

10 

3 

30 

90 

.62 

.01734 

-0.22 

1.2 

9 

4 

36 

144 

.84 

.05552 

-0.70 

1.5 

4 

5 

20 

100 

1.06 

.13198 

-1.66 

1.8 

4 

6 

24 

144 

1.28 

.259,98 

-3.26 

X 

112 

22 

1008 

-13.49 

1  112  1821 

at 

12.53147 

Ko  " 

11000  45501 

-  0.211,  Ub 

«  0.22 

26026 

<rM 

A1  “ 

22/182  - 

0.12088 

0.22 

x  ■  0.048, 

Use  0.04 

A2  - 

1  13  1121 

1 162  1008 1 

-  -0 

.27972 

y  - 

0.06482 

26026 

6*  • 

1.42060 

y 

X  ■ 

0.21607 

N  ■ 

149 

f(x) 

-  12.54453 

- 

4.73533 

79 

. . 


V 

■' 


I 


$ 


t 

V 

1 


% 

V 

> 


f 


B0  -  . 

98.51  182 

617.56  4550 

26026 

j  -  12.90343 

Bl  - 

30.72/182  - 

0.16879 

b2  “ 

1  13  98.51} 

1182  617.561 

26026 

-  -0.3804095 

x  “ 

0.22185 

f(7)  - 

12.92215 

- 

4.12123 

y  - 

0.06656 

°y  * 

1.23637 

N  - 

133 

1  - 

0.243,  Usa  0.26 

0.26  p 

-  0.058,  Usa  0.06 

C0  » 

I  93.2*  110 

1514.51  1958 

j.  -  13.34688 

9438 

Cl  - 

25.61/110 

•  0.23282 

Co  " 

|  11  93.24 

IllO  514.51 

-  -0.48705 

& 

0430 

x  ■ 

0.23901 

f  Of)  -  13.37470 

<TX  -  3.70544 

7  «  0.07170 

-  1.11163 

N  ■  124 

1  -  0.270,  Use  0.28 

*x 

.28  x  "  0.067,  Use  0.06 


!  I 


'U'i 


8l 


IX 


TABLE  19 


9  POINT  CURVE  PIT.  PROBLEM  3 


£lE 

°x 

Pactor 

AF 

t  * 

£*AF 

4 

-1,18 

.19468 

-2.60 

3.40 

3 

-0.90 

.07198 

-0.96 

8,04 

2 

-0.62 

.01734 

-0.23 

10.77 

•1 

-0.34 

.00164 

-0.02 

10.98 

0 

-0.06 

.00000 

0. 

14. 

1 

0.22 

.00029 

0. 

13. 

2 

0.50 

.00750 

-0.10 

12.90 

3 

0.78 

.04191 

-0.56 

9.44 

4 

1.06 

.13198 

-1.77 

7.23 

5 

1.34 

Not  Used 

I 

-6.24 

89.76 

1  89.76 

60 

n 

1446.06 

708  K 

it  97074 

u 

o 

2772 

1  Ji  4  f  \J  f  * f 

Di 

-  25.80/60  »  0.43000 

9  89 

.76 

d2 

-  60  446 

.06  •  1 

-0.49461 

x  *  0.43469 
f(x)  ■  13,36419 

ax  ■  3.675S7 
-  0.13041 


X£ 

x2S 

*=lt 

Pactor 

-13.60 

54.40 

-1.26 

.24592 

-24.12 

72.36 

-..98 

.09886 

-21.54 

43.08 

o 

• 

1 

.02770 

-10.98 

10.98 

-  .42 

.00378 

0. 

0. 

-  .14 

.00005 

13. 

13. 

.14 

.00005 

25.80 

51.60 

.42 

.00378 

28.32 

84.96 

.70 

.02770 

28.92 

115.68 

.98 

.09886 

25.80  446.06 


ay  •  1.10267 
N  -  123 

1  -  0.272,  Use  0.28 

°x 

.28  x  -  0.122,  Use  0.14 
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THE  1979  SAMUEL  S.  WILKS  MEMORIAL  MEDAL 
Frank  E.  Grubbs 

The  Samuel  S.  Wilks  Memorial  Medal  Award  was  initiated  in 
1964  by  the  US  Army  and  the  American  Statistical  Association, 
and  has  been  administered  for  the  Army  by  the  American  Statistical 
Association,  a  non-profit,  educational  and  scientific  society 
founded  140  years  ago  in  11839.  The  Wilks  Medal  and  Award  is 
given  each  year  to  a  statistician  -  and  a  top-notch  onel  -  and 
is  based  primarily  on  his  contributions  to  the  advancement  of 
scientific  or  technical  knowledge  in  Army  statistics,  ingenious 
application  of  such  knowledge,  or  successful  activity  in  the 
fostering  of  cooperative  scientific  matters  which  coincidentally 
benefit  the  Army,  the  Department  of  Defense,  the  US  Government, 
and  our  country  generally.  The  Award  consists  of  a  medal,  with 
a  profile  of  Professor  Wilks  and  the  name  of  the  Award  on  one 
side,  the  seal  of  the  American  Statistical  Association  and  the 
name  of  the  recipient  on  the  reverse  side,  and  a  citation  and 
honorarium  related  to  the  magnitude  of  the  Award  funds,  which 
were  generously  donated  by  Phillip  G.  Rust  of  the  Winnstead 
Plantation,  Thomasville,  Georgia.  Mr.  Rust  originally  stimulated 
the  interest  of  Sam  Wilks  in  distributional  properties  of  the 
"extreme  spread"  (bivariate  range),  a  measure. of  the  "accuracy" 
of  rifle  shot  on  a  target. 

These  annual  Array  Design  of  Experiments  Conference,  at  which 
the  Wilks  Medal  is  awarded  each  year,  are  sponsored  by  the  Army 
Mathematics  Steering  Committee  on  behalf  of  the  Office  of  the 
Chief  of  Research,  Development  and  Acquisition,  Department  of  the 


Previous  recipients  of  the  Samuel  S.  Wilks  Memorial  Medal 
include  John  W.  Tukey  of  Princeton  University  (1965) ,  Major 
General  Leslie  £.  Simon  (retired,  1966),  William  G.  Cochran  of 
Harvard  University  (1967),  Jerzy  Neyman  of  the  university  of 
California,  Berkeley  (1968)  ,  Jack  Youden  (deceased)  formerly 
of  the  National  Bureau  of  Standards  (1969),  George  W.  Snedecor 
(deceased)  formerly  of  lowa  State  University  (1970),  Harold 
Dodge  (deceased)  formerly  of  the  Bell  Telephone  Laboratories 
(1971),  George  E.  P.  Box  of  the  University  of  Wisconsin  (1972), 

H.  0.  Hartley  (1973),  this  year's  President  of  the  American 
Statistical  Association,  Cuthbert  Daniel,  private  statistical 
consultant  (1974) ,  Herbert  Solomon  of  Stanford  University  (1975) , 
Solomon  Kullback  of  George  Washington  University  (1976),  Churchill 
Eisenhart  of  the  National  Bureau  of  Standards  (1977) ,  and  William 
Kruskal  of  the  University  of  Chicago  (1978). 

This  brings  us  up  to  this  year,  for  which  the  competition 
for  the  Wilk's  Medal  turned  out  to  be  keen  indeed,  and  as  usual 
tho  "best  man  won".  The  members  of  the  1979  wilk's  Memorial  Medal 
Committee  consisted  of  individuals  skilled  in  the  art  of  arguing 
their  points  and  getting  their  best  views  in  the  minds  of  others  1 
They  were:  Chruchill  Eisenhart,  Fred  Frishman,  Frank  Grubbs 
(Chairman),  Bill  Kruskal,  Jeff  Kurkjian,  and  Frank  Proschan. 

They  had  the  job  of  concentrating  on  some  12  deserving  candidates 
from  many  nominees,  and  coming  up  with  their  best  selection  ac¬ 
cording  to  the  Wilk's  Medal  criteria. 
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The  1979  Wilks  Medalist/  like  Sam  Wilks,  was  born  in  Texas. 

He  received  his  B.  A.  in  Physics  (with  highest  honours)  in  1934 
from  the  University  of  Texas,  and  later  his  Ph.  D.  in  Mathematics 
from  Princeton  University  in  1940,  in  the  first  cohort  of  Wilks 
disciples.  Between  1934  and  1940  he  taught  as  a  graduate  assistant 
or  instructor  at  Brown  University,  the  University  of  Texas,  and 
Princeton  University,  while  also  during  that  period  he  made  the 
transition  from  Physics  through  Applied  Mathematics  to  the  great 
field  of  Mathematical  Statistics.  After  obtaining  his  Ph.  D.  in 
1940,  he  returned  to  the  University  of  Texas  as  Instructor  in  Ap¬ 
plied  Mathematics  and  Astronomy,  interrupting  his  academic  career 
to  join  the  Bureau  of  Labor  Statistics  in  1942,  and  in  1944  a 
project  of  the  Applied  Mathematics  Panel  of  the  National  Defense 
Research  Council  (I  believe  under  Sam  Wilks) .  Then  from  1945  un¬ 
til  1948  he  was  Professor  of  Mathematics  and  Statistics  at  Iowa 
State  University,  and  then  joined  the  RAND  Corporation,  where  he 
served  as  Deputy  Chief  of  the  Mathematics  Division  until  1955. 

At  this  point  in  time  the  entrepreneur  emerged  and  the  1979 
Wilks  Medalist  founded  the  General  Analysis  Corporation,  served 
as  its  President  until  1960,  when  it  merged  with  CEIR,  INc.  He 
then  became  a  Vice-President  of  CEIR  and  Manager  of  its  Western 
Division  until  1964,  when  he  next  went  to  Washington  as  Assistant 
Commissioner  of  Educational  Statistic*  in  ths  Office  of  Education. 
He  was  then  Director  of  the  Nationsl  Center  of  Educational  Sta¬ 
tistics  until  1967,  after  which  he  returned  to  the  West  Coast 
as  Professor  of  Administration  and  Director  of  the  Public  Policy 
Research  Organization  at  the  University  of  California,  Irvine. 


There  is  much,  much  to  say  about  this  scholar  and  gentleman 
concerning  his  great  contributions  to  the  field  of  statistics 
generally,  although  they  should  be  recorded  elsewhere.  We  should 
note,  however,  that  he  wrote  the  best  key  graduate  book  for  a 
solid  etatictics  course,  and  one  which  hen  trained  many  good 
statisticians.  (Introduction  to  the  Theory  of  Statistics).  He 
has  been  a  prolific  publisher  of  technical  papers  on  statistics, 
operations  research,  education  and  publio  policy  research.  He 
has  been  Presidents  of  both  the  Institute  of  Mathematical  Sta¬ 
tistics  and  the  Operations  Research  Society  of  America. 

A  long-term  friend  and  colleague,  George  w.  Brown,  told  me 
that  the  1979  Wilks  Medalist  is  an  "extraordinarily  and  decept¬ 
ively  quiet  man",  so  that  I  don't  think  he  would  win  an  award 
as  the  "most  talkative  statistician" 1  Vet,  he  has  exerted 
major  direct  and  indirect  Influences  on  an  enormous  number  of 
individuals.  And  he  has  had  many  separate  careers  actually, 
including  roles  as  a  professor,  a  think-tank  researcher,  an 

i 

operations  research  analyst,  an  administrator,  an  entrepreneur 
and  manager,  a  pioneering  public  servant,  founding  director  of 
an  important  research  organization,  and  distinguished  consultant 
and  advosor  to  universities  and  government  organizations. 

By  now  it  should  be  unmistakably  clear  that  we  are  referring 
to  none  other  than  Alexander  M.  Mood. 

The  citation  for  Alex  Mood  reads: 

"To  Alexander  M.  Mood  for  hie  many  significant  contributions 
to  the  theory  of  statistics,  an  outstanding  textbook  on  the  sub¬ 
ject,  his  extensive  applications  to  operations  research  and  sys¬ 
tems  analysis,  and  unique  statistical  assessments  of  education 
and  public  policy  research." 
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Variance  Reduction  in  Monte  Carlo  Simulation 


Mark  Brown 

Florida  State  Univaralty 

Herbert  Solomon 
Stanford  Univaralty 

Michael  A.  Stephana 
Simon  Fraear  Univaralty, B.  C, 


1.  Introduction, 

Monte  Carlo  simulation  ia  employed  in  a  large  variety  of  problems *  Fre 
quently,  one  is  interested  in  the  expectation  of  a  function  g(X1,...,XN) 
where  <  X,,  i  >  1  >  is  i.i.d.  with  known  distribution  F  and  N  is  a 
stopping  time  (often  e  oonatant).  Die  procedure  followed  is  to  generate 
a  large  number  of  samples  (X^,... i  *  1,2,...,M,  and  estimate 
the  expectation  of  interest  by 


1  n  _/y(l)  y  (l )  \ 

HA  8(xi  >  ‘ 

i-i  l 


An  intereating  aspeot  of  the  simulation  estimation  problem  is  that 
F  is  known.  Thus  functions  of  the  form  t  (F,X1,. .  .,Xjj)  oan  be  employed 
as  estimators,  \hile  in  statistical  estimation  problem  with  F  unknown 
i  cannot  be  computed  from  the  data  and  is  thus  not  considered  to  be  an 
estimator.  Thus  the  class  of  estimators  is  considerably  wider  in  Monte 
Carlo  problems. 

One  approach  available  to  reduce  the  variance  of  the  Monte  Carlo 
estimator  is  to  find  a  function  l(F,X1,...,XN)  with  the  same  expectation 
as  g,  and  with  smaller  variance.  Then  l  rather  than  g  is  averaged 
over  the  M  samples.  Of  course,  i  -  E^g  fits  thie  description  but  were 

Partially  supported  under  U.S.  Army  Research  Office  Grant  DAAG-29-77-G-0031 
and  issued  as  Technical  Report  No.  35. 
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it  directly  computable  one  would  not  need  to  simulate  in  the  first  place. 

Thus  an  important  requirement  of  I  is  that  it  be  simply  computable. 

We  illustrate  the  above  remarks  by  considering  the  problem  of  Monte 
Carlo  estimation  of  M(t)  -  3N(t)>  the  expected  number  of  renewals  in 
[0#t]  for  a  renewal  process  with  known  interarrival  time  distribution  F. 
Several  unbiased  estimators  which  ooupete  favorably  with  the  naive  estimator, 
N(t),  are  presented  and  studied. 

We  believe  that  our  approach  and  methodology,  although  only  applied  to 
renewal  function  estimation  in  thlt  paper,  can  ba  uaeful  In  a  large 
variety  of  Monte  Carlo  simulation  problems. 

£.  Assume  that  <  X^,  i  >  1  >  is  l.l.d.  with  odf  F  where  F(o)  *  0. 

Define  SQ  -  0,  Bn«  n  ■  l,2,..,,N(t)  ■Dux(m  Sn<t),  and 

1 

M(t)  -  EN(t),  t  >  0.  Sometimes  wa  oonsldsr  tbs  point  t  -  0  as  a  ranawal 
apoch.  In  thia  oast  wa  usa  NQ(t)  ■  M(t)+1  and  ^(t)  ■  M(t)+1.  Ihe 
renewal  age  at  time  t  is  dafined  by  A(t)  ■  Pr(A(t)«t)  -  ^(t) 

and  dF^)  ■  f(x)dM(t-x)  for  0  <  x  <  t,  thua  ^^t)  " 
for  0  <  x  <  t. 

Define 

1  if  fl1  <  t 
0  if  >  t  . 

Then  N(t)-Sbi  and  M(t)-BE  ^  -  f  F^ (t),  where  F(1^  is  the  Ith 
1  11 

convolution  of  F. 
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we  will  use 


To  estimate  F^(t)  » 

-  1(61|01.1)  -  . 

We  then  estimate  M(t )  by « 

oo  N  (t  )+l 

(1)  Mp(t )  -  ^  ^  *(4-8^)  . 

Since  Varfaft-S^))  ■  Var  (E^  1 9^))  <  Var  6^,  we  have  replaced 
each  component,  6^,  by  a  component  with  the  same  expectation  and 
■nailer  variance.  Intuitively  we  would  expect  that  If  we  reduce  the 
variability  at  eaoh  stage  (fiiven  the  past)  then  we  should  reduoe  the 
variability  of  the  overall  estimator.  However,  the  computation  of 
varlanoe  involves  covariance  terms,  and  if  these  are  increased  while 
variances  are  decreased  there  can  conceivably  be  an  increase  in  variance. 
Theorem  1  (below)  demonstrates  that  My(t)  dove  indeed  have  lower  variance 
than  M(t). 

Theorem  l.  Mp(t)  Is  an  unbiased  estimator  of  M(t)  and  Var  N(t)  -  Var  My(t) - 
B[2M(A(t ) )  -  F (A(t ) ) ]  >o,  with  striot  equality  if  F(t)  >  0. 


Before  proving  theorem  1  we  cement  that  the  reduction  in  variance 
is  unsatisfactorily  small  for  large  t.  If  u2  *  EX2  <  oo  then 
E[2M(A(t))  -F(A(t))]  -0(1)>  thus  Var  N(t)  and  Var  Mp(t)  are  of  the 
form  yt  +  0(1)  with  common  7,  and  we  improve  only  the  asymptotically 
negligible  0(1)  term.  Estimators  considered  in  later  sections  do 
considerably  better  for  large  t. 


i  Bfitrr  !■)"-“ U| 
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Proof  of  ahsorax  1.  Xxprssa  M^fc) 


p(t)  +Jt  p(t-x)dK(x)  «  Jt  P(t-x)dNQ(x)  . 


Thsn 


*^(4)  »J*  p(t-*)a^(*)  «  J*  ldfy(x)  .f*  F(t*)d)^(x) 
-  »^(t)  -  Jb  -  M(t) . 


«uW| 


wj(t)  -  Jt 

^  P(t-r)P(t-t)<l^(r)(JI^(«-r)  . 


+  2 


W*  svalutts  this  sxprsision  la  stvsrsl  stspsi 

fc  *  ft  ft 


(I)  ^  F^t-sOd^x)  -  J  P(t-x)dfy(x)  -  J ’  P(t-x)F(t-x)fl»^(x) 

-  M(t)  -  EP(A(t))  . 

(II)  F(t-r)P(t-s)  -  l-F(t-r)  -  F(t-s)  +  P(t-r)P(t-s)  . 


(Hi) 

(lv) 


2  JJ  ld^3(r)d^,(s-r)  -  2  Jt  M(t-r)<SI^(r)  -  2M(t)  +  2M(2)(t)  . 
-2  JJ  P(t-r  )dJ^ (r )d*k (*-**)■  "2  /  *  F(t-r)M(t-r)dl^(r) 

r  '<  a  •'r»o 


-2EM(A(t))  . 


92 


SfrW^h'iWW  , _ 


■*  ,4,  ?(t")a^trH(-r)  •  -*/*  Kt-r)a^(r) 


-  -2M(t)  . 


(vi)  a 


Jl  jr(t-r)F(t..)d^(r)di^(»-r)  -2  [' 


F(t-r)F(t-r)d*^(r) 


a»(A(t))  . 


Combining  (i)-(vi)  we  obtelm 


S^(t)  -  M(t)  +  2M(2)(t)  -  »(2H(A(t))  -  P(A(t)))  . 


furthermore 


0)  ra*(t)  -  it  f  *  ias(t)]2.  M(t)  *  e  ff  m(r)m(,.r) 

0  r<  a 

-  M(t)  +  2M(S)(t)  . 


M»u«  from  (g)  and  (3). 


V*r  H(t)  -  Vmr  M^t)  «  K[2M(A(t))  -  *<A(t»]  . 


Since 


M(i)  -  2M<«)-P(«)  -  p(.)  +  2  |  f(i)(,)>0j 

1x1  its  -  * 
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thus  El2M(A(t))-F(A(t)))  >o  for  »vi  t  and  i»  strictly  pofltive 
for  F(t)  >  0. 


3.  In  till*  Motion  we  assume  that  F  la  continuous.  The  curative 
hazard  H  is  defined  by  H(t)  -  -log  F(t).  Vtaen  F  la  abuolutely 
continuous  with  density  f  then  H(t)  -  /J  h(y)dy  tfiere  h  is  the 

hazard  function,  h(t)  ■  • 

Our  next  estimator  is  based  on  the  intuitive  idea  that 
E(dN(e)|past )  «  <SH(A(e)).  Thus  inetead  of  using  N(u)  -  /0  dM(s)  we 

try 

,  rt  N(t)  n(ty+i 

^(t)  -J  dH(A(s))  -  t«(*i^  +  H(A(t) )  -  £  \ 


Where  A  \\  (*»•«  a  A  b  •  min(a,b)). 

Kotethat  »<*)  -  |  et  while  ^(t)  -  8  V  B«*  \ 
by  H^,  and  ■  E(Hi|8^^)  ■  Fft-S^) 

The  process  M^t)  is  a  cumulative  process  in  the  sense  of  Smith 

[3],  Thus  (Smith  (33) 

Var  M^t)  -  J  E[H(X)  -  (S2l£l)X]2  , 


q 


vh.«  ,  -BX.  But  H(X)  -  -log  rot)  i.  «pon«tUU»  Jl.WHxit.4 


parameter  1,  thust 


where  p  la  the  oorrelation  coefficient  between  X  and  II (X)  and  <j 
la  the  variance  of  X.  Thua  M^t)  la  aayaptotically  better  than  N(t) 
for  p  >  \i/2u,  aaymntotieally  woree  than  N(t)  for  p  <  n/2<r. 

In  general  if  we  have  two  unblaaed  eatimatora  of  a  parameter, 
and  T2>  with  covariance  matrix  k,  then  the  minimum  variance  unblaaed 
eatimator  of  the  form  +  (l-a)T2  ia  the  one  with 

d-i 


k  £& 


The  variance  of  thia  eatimator  ia 


Hie  idea  now  ia  to  let  A  be  the  aaymptotic  oovarianoe  matrix  of 


(£iii, 

vs  yt 


and  to  employ  the  above  reault  to  obtain  an  unblaaed  eatimator  which 
improve*  on  both  M^t)  and  N(t)  for  large  t.  We  already  know  the 
0(t)  terma  for  Var  N(t)  and  Var  M^(t).  We  only  need  the  leading 
term  for  Cov(N(t),  M^t)).  Thia  ia  given  in  leaa&a  1  below. 

2 

Lemma  1.  If  a  ia  finite  then 

Cov(N(t),  M^(t))  «•  j  (2^  -  2£)  +  o(t)  . 


Proof. 


00 

Vwr(N(t)-MH(t))  -  Var  £  (®1-H(t-S1-1  A  ) ) 

-  1 1  Varlfti-  HCt-S^i  A  XL)lS1_;L]  -  *pMw)  -  *»<*>  "  M(t)  * 

Thu« 

M(t)  -  Var(N(t) -  Var  N(t)  +  Var(*^(t))  -  2Cov(N(t),fy(t))  , 


z  c 

±,i  J 


1  v'  *  TT  (1-P2)  • 
A.1  ^ 


Note  that  the  asymptotic  relative  savings  In  variance  is.  p  the 
square  of  the  correlation  ooefficient  between  X  and  H(X).  Summarizing: 
Theorem  2.  The  estimator 

M#(t)  -  (l-ffi)H(t)  +2£^,(t) 

is  an  unbiased  for  M(t)  with  variance 

(1“P2)  +  O(t) 

(p  is  the  correlation  coefficient  between  X  and  H(X)).  It  follows 
that: 


Var  N(t)  -  Var  M  (t)  2  .  . 

- S&rYrci — ^  3  p  +  o(l)  * 


Example;  Let  H(x)  *  x  ,  F(x)  -  e  .  Then, 


'  00  2 

e"x  dx 


^11  i 


2 

e"x  dx 


p  f 00  2 

EX*  -  2  xe"X  dx  =  1  , 

Jo 


2  ,  «  4-n  1  r 4  -x  ,  ,  u  1  /  n  2  n  c 

-  -  i-ir *  tt  > p  *  ?  U2*  e  dXHl]  a  2? a  2  ter » p  =  ctitstt  -  *915 


rr/  r" 


Thus  in  this  cue  (Wei bull  with  shape  parameter  2)  the  unbiased  estimator 
M  (t)  hu  an  asymptotic  relative  reduction  in  risk  over  N(t)  of 
91. 5  percent.  || 

Integration  by  parts  shows  that 


H(x)F(x)dx  ; 


since  H(x)  -  -log  F(x)  the  integral  can  probably  be  given  an  enthropy 
interpretation.  Also  p  -  ^  EH(X)  where  H(x)  ■  f*  H(z)dz.  This  is 
true  since 

J°°  H(x)F(x)dx  -  J  H(x)XIx>x  <3x  *=  E  J  H(x)lx>jtix  -  E  J  H(x)dx  «  EH(X). 

Note  that  both  p  and  are  invariant  under  a  change  of  time  scale, 
t  •*  ct,  c  >  0. 


4.  In  section  3  we  estimated  M(t)  by  a  weighted  average  of  N(t)  -  E 

N(t)+1  1 

and  MpCt)  a  T.  H((t-S1-1)  A  Xj).  Now  we  apply  the  same  idea  but 

stagewlse.  At  stage  i,  having  observed  N(t)  adds  the 

component  8^  «*  1^  <  ,  while  Mp(t)  adds  ■  K((t-S^-1)  A  X^. 

Each  of  5^,  are  conditionally  (given  )  unbiued  for  F(t-Si ^ ) 
and  unconditionally  unbiued  for  F^(t).  The  approach  we  now  follow  is 
to  use  the  weighted  average  of  8i  and  which  hu  smallest  conditional 
variance  given  X1,...,X^_1. 


Define  Fa  ■  F(t-Sif>1),  ^  -  Hft-S^).  Then: 


V«(6tISi.i)  -  Fj-jf 

C0V(6l,HilSi-l)  ’  ^ 

Var  (Ht  1 81.1 )  -  Fi+yj_  (F1-2C1 )  . 
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The  minimum  conditional  variance  (given  X^,...,X.  ^5  unbiased  linear 
coaibi nation  is  then: 


CiPi 

» -  4r)»i 


C  F 
1  i 


I 

l,J 

.■i 

■■■« 

i 

j 


ae  corresponding  estimator  of  M(t)  Is: 


M^t.)  =  N(t) 


N(t)+1  H(t-S1-;L)F(t-Siml) 


F(t-BlMlJ 


<VV 


We  do  not  know  how  M^(t)  compares  with  the  other  estimators  we 
have  looked  at.  The  variance  of  an  estimator  of  the  form  £  is 
£  Var  +  2  ^  £  Cov(K^>Kj)}  was  chosen  from  among  a  class  of 
estimators  £  to  minimize  £  Var  K^.  However  we  know  very  little 
shout  Cov(L^,Lj ).  This  latter  quantity  must  be  shown  to  be  suitably 
small  in  order  to  demonstrate  that  M^t)  has  desirable  variance 
properties . 
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5.  We  next  consider  an  unbiased  estimator  with  asymptotic  variance 
0(1 ).  Thus  it  asymptotically  enjoy*  a  100  percent  reduction  in 
variance  over  H(t). 

As  is  well  known  N(t)+1  is  a  stopping  time  and  thus  by  Wald's 

i 

identity: 


Thus 


N(t)+1 


M(t) 


h&SL.i 

W 


is  unbiased  for  M(t).  Hbw  Var(8jj^+^)  •  Var(t+Z(t))  ■  Var  Z(t), 
where  Z(t)  is  the  forward  recurrence  time  at  t.  If  \x^ 
then  Var  Z(t)  converges  to 


00  ( 


Thus 


W.  Ug  W«-3d| 

i-TT'-hr* 


law 


Var  M(t) 


‘HAW, 


,  2 
-3u2 


12u 


and  is  thus  0(1  )> 
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SMALL  SAMPLE  SIZE  EFFECTS  ON 
TOLERANCE  LIMITS,  EXCEEDANCES 

JAMES  R.  KNAUB,  JR. 

ARMY  MATERIEL  TEST  AND  EVALUATION  DIRECTORATE 
US  ARMY  WHITE  SANDS  MISSILE  RANGE 
WHITE  SANDS  MISSILE  RANGE,  NEW  MEXICO 

ABSTRACT  -  Tolerance  Limits,  exceedances  and  includances, 
are  useful  indications  of  the  adequacy  of  a  sample  size. 
However,  for  very  small  sample  sizes,  such  results  become  very 
sensitive  and  may  require  a  thorough  analysis  before  con¬ 
cluding  whether  sample  sizes  are  adequate.  Some  measures  of 
sensitivity  are  investigated  in  this  paper. 

1.  Introduction  -  When  dealing  with  missile  systems  or  any 
other  materiel  which  consists  of  a  relatively  small  number 
of  very  expensive  items  which  will  not  survive  a  test,  one 
would  not  like  to  destroy  any  more  materiel  than  necessary, 
so  sample  sizes  are  made  small.  Time  may  also  be  a  factor 
in  keeping  sample  sizes  small.  When  testing  to  specified 
test  objectives,  however,  small  sample  sizes  cause  large  un¬ 
certainties  in  the  results  obtained.  In  hypothesis  testing, 

for  example,  a  small  sample  size  means  that  the  power  of  the 
test  is  low,  and  therefore  one's  ability  to  discriminate  be¬ 
tween  an  untrue  null  hypothesis  and  a  true  alternative  hypo¬ 
thesis  may  be  low  even  when  the  two  hypotheses  are  very  dif¬ 
ferent.  Although  the  power  of  a  test  is  very  important  in 
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missile  work,  as  it  shows  the  sensitivity  of  results  to  small 
sample  sizes,  it  is  often  ignored.  An  example  is  the  use  of 
Wilcoxon'.s  Rank  Sum  Test  which  is  often  used  with  no  power 
calculated,  even  though  it  is  easily  obtainable  from  a  paper 
written  by  E.  L.  Lehmann  in  1953  (Lehmann,  "The  Power  of  Rank 
Tests,"  Annals  of  Mathematical  Statistics,  24  (1953),  23-43). 

Any  measure  of  sensitivity  which  provides  the  likelihood 
of  confusing  one  result  for  another  would  be  analogous,  to  a 
degree,  to  power.  The  central  question  is,  "Is  the  sample 
size  sufficient  to  reduce  to  an  acceptable  level,  the  risk 
of  saying  that  more  is  known  than  actually  is  known?" 

From  Gumbel,  Statistics  of  Extremes ,  pages  97  and  103-104 
(see  also  Hoel,  Introduction  to  Mathematical  statistics,  pages 
274-277),  the  following  equation  is  produced  (due  to  S.S.  Wilks) 


P  *  l-ny”"1  +  (n-l)Yn 


where  P  i  P[at  least  100^%  of  the  population  is  between  the 
smallest  and  largest  observation  of  the  sample  of  size  n]. 

A  first  approximation  (which  appears  to  be  low  for  small  sample 


sizes)  is  given  as 


nu1  +  iM 1 


The  previous  equation  can  be  solved  iteratively  from  here 
using  small  increments. 

This  is  useful  information.  However,  for  very  small 
sample  sizes,  perhaps  a  measure  of  sensitivity  as  described 
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earlier  may  b«  desired.  To  maka  one  further  analogy,  when 
sample  sizes  are  very  large,  point  estimates  may  be  suffi- 
oient,  hpwever,  when  sample  sizes  are  small,  lower  confidence 
limits  are  needed  in  order  to  avoid  misleading  decision 
makers.  What  is  to  follow  is  based  upon  the  theory  of  exceed¬ 
ances.  (See  Bradley,  Distribution-Free  Statistical  Tests, 
pages  216-218.) 

2.  A  Measure  of  Sensitivity  -  Consider  an  initial  sample  X, 
made  up  of  n  observations,  ranked  from  r  equal  1  to  n,  from 
which  one  wishes  to  make  predictions  about  the  next  m  obser¬ 
vations  in  a  sample,  Y.  If  b  represents  the  number  of  observations  in  Y 
which  have  values  lower  than  the  rth  lowest,  value  in  X,  then  (from  Bradley, 
pages  216-218), 


P[pxactly  b  of  the  Y's  will  be  <Xr] 


.r-l+b,  .n+m-r-bv 
l  r-l  M  n-r  • 
.n+m. 

1  n  1  . 


l 

Letting  lim  -jjj-and  p£ 


•,  where  P0<Pi/one  can  in¬ 


vestigate  the  case  of  n,  a  finite  sample  size,  but  n+m,  the 
infinite  population  size.  Now  let  p  be  used  to  represent  a 
measure  of  sensitivity  for  tolerance  limits,  in  some  ways 
analogous  to,  but  not  the  same  as,  the  probability  of  a  type 
II  error.  Here, 

p  s  »[100p] t  or  more  Y'b  <  Xr] 

:  P[100po%  or  fcwor  Y'e  <Xr]. 

6,  here,  is  not  a  probability,  but  a  ratio  of  probabilities. 


tmm  my  ■ 


It  shows  the  relative  probability  of  having  an  unacceptable 
(100p1%)  percent  of  the  population  values  for  a  certain  para¬ 
meter  (e.g.,  missile  lethality),  or  more,  fall  below  the  rth 
lowest  value  in  the  X  sample,  as  compared  to  a  hoped  for  100pQ% 
or  fewer  members  of  the  population  falling  in  that  oategory. 
Therefore,  the  smaller  0,  the  more  sensitive  the  results  are 
to  implying  more  is  known  than  can  be  known  from  a  certain 
sample  size,  (i.e.,  the  smaller  0,  the  better.) 

The  following  briefly  sketohes  the  derivation  of  0: 
m 


£  ,r-l+j.  .n+m-r-j. 

0  ■  lim  j-bi {  r-1  '  {  n-r _ _ 

It H—  Dg 

m 

E  [ (r-l+j ) (r-2+j ) , . . <  j+1) (n+m-r-j ) (n-Hn-r-j-1) . . . (m- j+1) ] 

-  lim  j«b,i _ _ 

ro*«  fcu 

j;'  [  (r-l+i)  (r-2+i) . . .  (i+1)  (rrtir*- r-i)  (n+m-r-i-1) . . .  (m-i+1)  ] 
i-o 

For  m  large,  b^^Pj^m  and  k0*p0m,  *°  8um  in  numerator  from 
(Ejm)  r”1[m(l-Ej)  ]n“r  to  mn”1(n-r)!,  and  in  the  denominator  from 
(r-1)  Jm”^  .to  (pQm)  r”1[m(l-p0)  ]n"r .  Since  there  are  an  infin¬ 
ite  number  of  terms,  these  summations  become  integrals.  For 

r-l,  n-5,  the  numerator  beoomea  lim  Jf  (3«i-j)  (2+m-j)  (l-m-j)dj, 

”**  P,m 

which  makes  the  integral  approach  m  .  For  r»2,  n-5,  the 

/(m-jrdj- 
Pim  m 

m  1  is  3 

integral  is  / { j+1) (3+m-j ) (2+ra-j ) (1+m-j )dj  +  /j(m-j)  dj. 
p^m 


106 


In  general  then,  the  Integrals  will  be  of  the  form 
/jr”1(m-j)n"rdj. 

The  only  difference  between  numerator  and  denominator  are  the 
limits  of  the  integration. 

For  r»l,  n*S,  one  has 

6  -  l-p1+2{p12-l)+2(l-p13)+p14-l+|(l-p15)> 
po  2po  +2po  -po  +Spo 

As  an  example,  letting  Pj“0.3  and  po*0.2,  e(r-l,  n-5 ,  pQa0.2, 
p,,  "0. 3)  “0.  42.  This  means  that  the  probability  that  30%  or 
more  of  the  population  will  have  values  below  the  lowest  in 
the  sample  of  size  5  is  42%  of  the  probability  that  20%  or 
leas  of  the  population  will  be  that  low.  If  one  does  not  wish 
to  tolerate  having  more  than  20%  of  the  population  that  low, 
then  there  is  cause  for  a  great  deal  of  alarm,  especially  if 
30%  is  an  unreasonable  alternative. 

One  can  determine  the  exact  probability  of  having  lOOaft 
or  more  of  the  population  fall  below  the  rth  value  of  the 
sample,  and  also  the  exact  probability  of  having  100a%  or 
less  of  the  population  fall  there.  Call  the  first  prob¬ 
ability  x  and  the  second  y,  Then,  one  has 
x+y  -  1,  and 
6 (po“P1»100a%)  «  ^ 

Therefore,  for  such  a  0,  x  and  y, 
x  "  TTb  and  y  "  ITb 
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As  an  oxample,  B(r«l,  n*5,  pQ»0.2,  p.,-0. 2)  «  0.55, 

P[20l  or  mors  of  tha  Y'e  <  x^]  ■  0.36,  and  P[206  or  less  of 

the  Y's  <  X^3  ■  0.64. 

For  r-2,  n«5,  one  has 

6  -  Ia“pl2)>.  +  pl3  “  1+fu-Pi4)  +  , 

r~2 - - - 

Ipo  po  +  7po  '  5po 

As  an  example,  for  po-0.4  tested  against  the  alternative  that 
Pj-0.5,  b  »  0.28. 

As  can  be  seen  from  the  above,  one  may  calculate,  a  priori, 
what  ranges  of  values  of  n  and  r  may  be  used  in  order  to  have 
an  adequate  degree  of  faith  in  the  accuracy  of  results.  A 
major  advantage  in  this  approach  is  that  one  may  use  r>l. 

Often  one  aan  not  obtain  an  adequate  sample  size  without  a 
large  probability  that  at  least  one  of  the  observations  will 
be  lower  than  a  value  toward  which  one  would  like  to  test. 

3.  A  More  Subjective  Approach  to  Sensitivity.  When  planning 
sample  sizes  for  a  test,  there  is  an  additional  approach  which 
may  be  helpful.  Consider  Danzigar  and  Davis,  "Tables  of  Dis¬ 
tribution-Free  Tolerance  Limits,"  Annals  of  Mathematical  Sta¬ 
tistics  ,  35  (1964),  1361-1365.  From  Danziger  and  Davis, 

"The  probability  that  NQ  of  the  Y^'s  lie  above  Xr  is  given  byi 

n/w  *  m  ,N  +n-r  wN-No  1  r_i)  /  (N  1  n.  „ 

P<V  (  °H  )(  N-N  '  » 

o  o 

Here,  i  denotes  a  ranking  of  the  Y  sample,  N  is  what  has  been 
labelled  m,  and  No  is  the  number  of  Y^'s  above  X^,  where  b 


has  tossn  ussd  earlier  to  represent  the  number  of  them  below 
it. 

From  the  table  by  Danziger  and  Davis,  with  probability  at 
least  0.75,  at  least  0.752  of  a  population  will  lie  above  the 
second  lowest  value  in  a  sample  of  ten.  with  probability  at 
least  0.75,  however,  the  proportion  of  the  population  above 
the  third  lowest  in  a  sample  of  ten  is  only  at  least  0.644. 
Therefore,  if  X2-X3  is  very  small,  then  a  sample  of  size  ten 
is  not  adequate  in  that  it  can  not  distinguish  very  well  be¬ 
tween  0.752  and  0.644  as  lower  probability  bounds  to  the  pro¬ 
portion  of  the  population  values  above  such  a  point.  Examin¬ 
ing  the  table  by  Danziger  and  Davis  shows  that,  in  the  oases 
shown,  this  form  of  sensitivity,  as  are  most  forms  of  sensi¬ 
tivity,  is  extremely  sample  size  dependent. 

As  a  practical  example  of  how  this  approach  could  be  used, 
consider  the  case  of  determining  whether  targets  are  detected 
by  a  certain  range.  Imagine  that  the  same  number  of  targets 
are  detected  before  a  critical  range  in  each  of  two  tests 
using  equal  sample  sizes.  In  one  case,  say  all  (or  many)  of 
the  detections  before  the  critical  range  were  just  barely 
before  it,  but  in  the  other  case,  they  were  very  early.  Thus, 
a  slight  change  in  critical  range  would  cause  a  large  change 
in  the  proportion  of  the  former  population  values  believed  to 
be  above  this  point,  but  no  change  in  the  latter  case.  (This 
situation  is  expected  when  dealing  with  rank  procedures.) 


4.  Conclusion  -  When  analysing  data,  ona  must  ba  particularly 
caraful  in  dealing  with  small  sample  sizes.  It  is  highly 
desirable  to  obtain  every  bit  of  information  possible  from 
such  samples,  but  it  is  equally  desirable  to  avoid  claiming 
more  knowledge  of  the  population  than  the  sample  can  actually 
provide.  Balancing  cost  considerations  against  information 
obtainable  can  only  ba  accomplished  by  careful  consideration 
of  all  aspects.  Tolerance  limits  and  exceedances  haye  a  role 
in  such  considerations. 
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ABSTRACT .  There  Is  a  phenomenon  that  appears  to  exist  in  the  portrayal 
of  measured  data  for  nearly  all  types  of  experiments .  It  is  observed  that 
when  experimental  data  points  are  plotted  as  a  function  of  a  variable,  the 
data  points  are  portrayed  periodically  rather  than  randomly  about  the  theo¬ 
retical  curve.  This  is  in  contrast  to  an  expected  random  distribution  of 
the  measured  data.  Although  not  always  well  defined,  two  distinct  periodic 
characteristics  can  be  pointed  out:  The  first  is  a  sinusoidal  characteristic 
that  appears  to  be  cyclic  in  10-12  measured  events,  and  the  second  is  the 
"cusp"  characteristic  that  appears  to  be  cyclic  in  about  every  three  to 
four  measurements.  A  method,  which  is  based  on  a  heuristic  equation  that 
relates  a  "periodic"  probability  of  the  arithmetic  deviation  with  the  associated 
measurement,  generates  data  points  which  are  in  agreement  with  aome  published 
experimental  values. 

I.  INTRODUCTION.  There  is  a  phenomenon  thnt  appears  to  exist  in  the 
portrayal  of  measured  data  for  nearly  all  typeB  of. experiments.  The  phenomenon 
is  that  when  experimental  data  points  are  plotted  as  a  function  of  a  variable, 
the  points  are  manifested  in  a  periodic  fashion  about  some  smoothly  drawn  curve. 
The  periodic  appearance  of  the  data  points  is  in  contrast  to  an  expected  random 
distribution  of  the  measured  data.  The  periodicity,  althoughnot  always  well 
defihed,  can  be  seen  upon  examination  of  such  plots  in  various  technical 
journals.  Two  distinct  periodic  characteristics  canba  pointed  out.  The  first 
is  a  sinusoidal  characteristic  that  appears  to  be  cyclic  in  10-12  measured 
events  (1,2,  3,4).  The  second  is  the  "cusp"  characteristic  (5)  that  appears 
to  be  cyclic  in  about  every  three  measurements,  (6,  7,  8,  9)  whereas  other 
curves  appear  to  be  a  combination  of  both  characteristics  (10).  The  cited 
references  are  specific  examples  selected  from  the  literature  that  clearly 
exhibit  the  periodicity.  However,  most  other  figures  show  data  points, 
thnt  exhibit  the  periodic  deviations  of  the  data  points  less  clearly ,  but  nearly 
all  of  the  experimental  plotted  data  points  show  the  phenomenon  regardless 
of  the  physical  parameters  that  were  measured.  This  phenomemon  is  based  on 
observation  devoid  of  a  priori  concepts  of  the  randomness  of  events  and  asso¬ 
ciated  measurements.  ~it  is  thus  concluded  that  there  is  a  non-random  re¬ 
lationship  between  the  experimentally  measured  data  and  the  sequence  of  meas¬ 
uring  events.  In  order  to  develop  a  mathematical  treatment  for  the  perlodio- 
dlcity  of  experimentally  measured  data,  the  following  postulated  is  made: 
If  for  a  well-ordered  experiment ,  the  experimentally  measured  data  are  taken  in 
an  identical  manner  at  equal  increments  of  the  independent  paramoter. 
the  data  points  will  be  periodic  as  a  function  of  the  (measuring)  events  about 
the  average  curve.  Thus,  the  experimentally  measured' data  can  be  determined 
from  the  probability  of  the  associated  meaouring  event;  in  conjunction  with 


the  equation  i  r  the  experiment,  It  Is,  In  principle,  possible  to  predict 
the  experimental  data  points  as  a  function  of  measurements. 

The  purpose  of  this  paper  is  to  present  a  heuristic  method  that  generates 
a  periodic  form  for  data  points  which  are  In  general  agreement  with  published 
experimental  values.  Three  examples  In  which  the  "periodic"  data  points 
exhibit  close  agreement  with  published  experimental  data  points  will  be 
shown. 


II.  DEVELOPMENT.  The  deviation  of  a  datum  point  from  theoretical 
curve  ^considered  to  be  the  mes  0  Is,  by  definition,  the  arithmetic  deviation. 
The  arithmetic  deviation  of  a  measured  value  of  a  physical  property  of 
an  experiment  Is  dependent  upon  the  experimental  procedures,  and  if  enough 
measurements  are  made,  the  distribution  of  the  arithmetic  deviations  will 
follow  the  Gaussian  curve.  From  the  Gaussian  distribution  curve,  It  can 
be  readily  seen  that  the  arithmetic  deviation  is  proportional  tp^the  standard 
deviation.  Tills  proportionality  factor  is  the  only  variable  In  the  Gaussian 
distribution  equation  and  thus  let  a  key  parameter  for  the  comprehension  of 
this  paper.  The  proportionality  f Actor,  derived  as  a  function  of  the 
probability  of  the  Gaussian  diatrlb.  ion  equation,  will  also  be  derived 
from  a  heurlstlcally  developed  periodic  "pseudo-Gaussian"  distribution  for 
the  generation  of  periodic  distribution  of  the  data  points. 

The  Gaua8ian  distribution  equation  is 

pm  *  l/(o/2n )exp[-(i»-M)2/2cr2]  (1) 

where  is  the  probability  of  the  measured  value,  m  la  the  measured  value,  ■ 

M  la  the  mean  and  o  la  the  standard  deviation. 

Let  m  -  M±tar  (2) 


where  kcr  is  the  arithmetic  deviation  and  k,  the  multiplier  of o  ,  is  defined 
as  the  arithmetic  deviation  coefficient.  By  substitution, 


Pm  -  1/  (o/5ir  )exp(-k2/2  ) . 

(3) 

Now  pm  is  normalized  so  that  the  area  under  the  curve  is  unity  and  thus 
pm  ■  0.399/0  when  m  **  M,  the  peak  of  the  Gaussian.  However,  by  lefting 

Pc  *  pm(o/2n),  a  comparative  probability  is  obtained  such  that  r_  ■  1  when 
k<-0,i.e.,m*<K. 

Now 

Pc  -  exp(-k2/2) 

(4) 

and 

k  «  +  /-2  In  Pc  . 

<5> 

Eq.  (5)  can  be 

rewritten  as 

k  -  ±/2I 

(6) 

dtfV  'M&Xi jfptttMF  Wdrf law •  -■-**** 
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where  I  *  -In  Pc  and  is  a  form  of  the  self-information  equation  (11)  of  the 
measurement  which  is  associated  with  the  arithmetic  deviation  coefficient. 
Thus,  if  the  comparative  probability  associated  with  a  measurement  can 
be  determined,  the  arithmetic  deviation  coefficient,  k,  of  that  measurement 
can  be  calculated  from  Eq<  (5).  Since o  is  a  constant,  k  is  proportional  co 
the  arithmetic  deviation* 

Since  the  portrayal  of  the  experimentally  measured  data  indicates  that 
the  measured  values  are  a  periodic  function  of  the  measurement  and  since  the 
comparative  equation  developed  from  the  Gaussian  equation  is  not  periodic,  a 
heuristic  periodic  expression  which  closely  matches  the  Gaussian  distribution 
(to  about  t  2.5o)  was  developed  for  the  probability  of  the  measured  value  as 
a  function  of  measurement* 

The  empirical  equation  is 

PN  -  cos26N  (7) 

where  is  the  probability  associated  with  measurement  N, 

N  “  1,2,3,  and 

eN  -  30  %  +  8/JkJI  (8) 

where  -•<  Kjq<+*>  and  is  dependent  upon  N.  This  dependence  will  be  discussed 
later. 


.  Comparisons  of  the  probability  values  of  Eqna.  (A)  and  (7)  are  shown  in 
Fig.  1  for  KN  and  k  in  the  range  -3.00  <k  *KN<+3.00.  The  comparison  shows 
Eq.  (7)  la  an  approximation  to  the  Gaussian  to  about  ±  1.75 o  and  a  reasonable 
approximation  from  ±  1.75®  to  about  ±  2* 5a*  Whan  P«  ■  0,  Kjj  “  ±  2.3723..., 
and  for  the  comparable  value  of  ±  2.57230  nearly  99$  of  the  measured  data 
points  would  be  Included.  An  extension  of  Fig.  1  would  show  that  Pc  would 
asymptotically  approach  sero,  whereas  would  oscillate  in  a  cosine 
squared  manner  periodically  in  about  every  5.2  Kp  but  the  period  would 
Increase  with  increasing  (and  decreasing)  K^. 

...  The  term  K«  1b  selected  such  that  %  ■  ai  +  (W-lKoo-a^)  where  is 
heurl8tlcally  selected  for  P^  associated  with  N*l,  the  first  measured  event, 
and  <*2  is  similarly  selected  for  P2  associated  with  N-2,  the  second  measured 
event.  Note  that  is  not  necessarily  an  integer  associated  with  the 
corresponding  Nth  measurement  and  thus  (°2“ai)»  may  nor  mry  not  be  an  lntsger. 

In  the  empirical  expression  for  (Eq.  7)  the  standard  deviation 
coefficient  per  measurement  does  not  appear.  However,  since  P  and  P^  serve 
comparable  functions,  and  since  k  can  be  determined  from  Pc,  it  is  postulated 
that  the  arithmetic  deviation  coefficient  per  measurement,  k^,  can  be 
determined  from  ?N  by  a  similar  expression  as  Eq.  5.  The  expression  is 


kj,]  “  ±^-2  In  Pjj  , 


(9) 
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where  kN  is  a  function  of  K„  through  Bqs.  (9)  and  (8)  in  a  transcendental 
fashion. 

The  measured  value,  m,  is  now  expressed  as 

m  “  M  +  kflOg  (10) 

for  each  measurement  H,  where  is  a  constant  and  defined  as  the  experi¬ 
mental  standard  deviation.  Thus  if  M  and  erg  are  known,  m  can  be  determined 
as  a  function  of  measuring  events. 

The  polarity  of  k^  is  eelected  as  either  +  or  -  for  the  cusp 
portrayal  of  the  data  points,  whereas  the  polarity  Is  alternating  in  sign 
(+  and  -)  corresponding  to  the  alternating  periods  of  PN  for  the  sinusoidal 
portrayals. 

Eq.  (10)  is  also  a  simplified  form  of  an  equation  for  a  specific 
physical  phenomenon  when  the  experimentally  measured  data  are  plotted  as 
a  function  of  the  Independent  parameter.  Formally,  each  measured  value 
will  be 


Y  -  F(X)  ±  ktfjg  (11) 

where  Y  is  the  measured  value,  F(X)  is  the  aquation  governing  the  physical 
phenomenon  and  k^og  is  the  arithmetic  deviation.  Thus  assuming  for  any 
experiment  the  equation  F(X)  and  the  dependent  constants.  #re  known,  the 
evaluation  of  F(X)  as  a  function  of  the  independent  variable  would  be  the 
mean  and  corresponds  to  the  theoretical  curve.  The  arithmetic  deviation, 
hence  Y,  can  be  made  as  a  function  of  measuring  events  If  an  _a  priori 
determination  of  a£  can  ba  made.  ” 

III.  EXAMPLES.  Eq.  (11)  shows  that  experimentally  measured  data  are 
periodic  as  a  function  of  measurement  events  about  some  smooth  curve 
governed  by*the  physical  equation.  If  F(X)  is  well-behaved  and  If  Og 
can  he  a  priori  determined  (*t  leaet  heurlatically  if  not  by  other  means) 
the  experimental  data  points  should  ba.  In  principle,  predictive.  The 
predictlvenaea  of  the  method  was  not  subjected  to  experimental  verification. 
The  reason  la  obvious  since  there  can  be  no  a  priori  determination  of  Og 
and  kN  for  Eq.  (11)  for  an  experiment.  However,  the  published  data  points 
of  three  experiments  have  baen  closely  duplicated  bv  the  described  method. 
Three  nxamplea  Indicate  a  posteriori  verification  that  experimentally 
measured  data  are  periodTc  as  well  as  predictive.  (The  calculations  ware 
accomplished  with  a  pocket  calculator,  and  thus  the  duplication  of  the  data 
points  were  not  optimised.) 

The  first  example  is  a  relative  easy  one:  The  sinusoidal  data  portrayal 
of  Ref.  1.  The  aquation  of  the  line  was  determined  to  be 

F(X)  -  i(t)  -  9.40  -  0.49t  (12) 
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where  l(t)  is  the  relative  transient  current  and  t  is  in  units  of  equal 
increments  of  time  corresponding  to  the  time  of  1.37  nsec  (average)  between 
'ei  H  measured  datum  point.  The  first  point  was  at  2.74  nsec  (or  the  3rd 
experimental  point)  which  selected  for  the  N-l.  Note  that  the  logarithmic 
representation  of  the  ordinate  was  portrayed  in  linear  unite.  The  terms 
oi,  «2  and0e  for  this  example  were  heurlstically  chosen  which  generated 
the  points  that  were  in  agreement  with  the  experimental  points.  The  terms 
and  generated  points  are  ^listed  in  Table  I  and  the  curve  is  shown  in  Fig.  2. 
The  generated  data  points  are  nearly  in  exact  agreement. 

Table  I.  Constants  and  "periodic"  data  points  generated  as  a  function 
of  measurement  for  th?  sinusoidal  example  for  Ref.  1. 

F(X)  «  i(t)  -  9.40  -  0.49t 


°1  " 

0.00.  a  2 

•1.00.0  g. 

-  0.05 

N 

% 

PN 

kN 

kN°E 

i(t) 

i(t)+ktfJE 

1 

0 

1.000 

0.00 

0.00 

8.06 

8.06 

2 

1 

0.607 

-0.98 

-0.05 

7.39 

7.34 

3 

2 

0.103 

-2.15 

-0.11 

6.71 

6.60 

4 

3 

0.057 

-2.37 

-0.12 

6.04 

5.82 

5 

4 

0.517 

-1.15 

-0.06 

5.37 

5.33 

6 

5 

0.956 

-0.30 

-0.02 

4.70 

4.68 

7 

6 

0.888 

+0.49 

+0.03 

4.03 

4.06 

8 

7 

0.393 

+1.37 

+0.07 

3.36 

3.43 

9 

8 

0.016 

+2.87 

+0.14 

2.69 

2.83 

10 

9 

0.165 

+1.90 

+0.10 

2*02 

2.12 

The  second  example  is  to  duplicate  Ref.  6  for  the  cusp  periodicity.  The 
equation  of  the  line  is  F(X)  ■  R(t)  •  98.0  where  R(t)  is  the  counting  rate  and 
is  a  constant  value  as  a  function  of  time.  The  measurements  were  made! in 
units  of  equal  increments  of  time  of  200  seconds  (average)  between  each 
datum  point.  The  first  point  waB  at  600  seconds  (or  the  fourth  experimental 
point)  which  was  selected  for  ftp  Again  ap  a 2  an<*  3  r  were  heurlstically 
chosen.  The  constants  and  the  calculated  values  are  listed  in  Table  II 
and  the  "predictive"  points  are  shown  in  Fig.  3(a)  by  the  solid  dots. 

The  open  circles  are  the  relative  positions  of  the  experimental  data 
points  as  estimated  from  Ref.  6  with  which  the  predictive  points  are 
not  in  agreement.  The  first  twelve  predictive  points  are  nearly  in  exact 
agreement  with  the  experimental  points  numbers  4  through  15.  However, 
the  13th  predictive  point  must  be  moved  to  the  18th  experimental  point 
for  the  cusp  periodicity  to  agree  with  increasing  measurements.  With 
the  exceptions  of  experimental  points  16.  17  and  30.  the  other  points 
are  either  in  nearly  exact  or  close  agreement. 


Table  II.  Constanta  and  "periodic"  data  points  generated  as  a 
function  of  measurement  for  the  cusp  example  for  Ref.  6. 

F<X)  -R<t)  -  98.0 


oi 2  m  •  00  j  a  £  **  3i00^o  £  •  0*8 


R(t)+kjja£ 


1 

1 

0.607 

-0.98 

-0.78 

97.22 

2 

3 

0.067 

-2.39 

-1.91 

96.09 

3 

5 

0.956 

-0.30 

-0.24 

97.76 

4 

7 

0.393 

-1.37 

-1.10 

96.90 

5 

9 

0.165 

-1.90 

-1.52  - 

96.48 

6 

11 

0.996 

-0.09 

-0.07 

97.93 

7 

13 

0.268 

-1.62 

-1.30 

96.70 

8 

15 

0.265 

-1.63 

-1.30 

96.70 

9 

17 

0.997 

-0.07 

-0.05 

97.95 

10 

19 

0.180 

-1.85 

-1.48 

96.52 

11 

21 

0.357 

-1.44 

-1.15 

96.85 

12 

23 

0.979 

-0.21 

-0.17 

97.83 

13 

25 

0,117 

-2.07 

-1.66 

96.34 

14 

27 

0.440 

-1,28 

-1.02 

96.98 

15 

29 

0.949 

-0132 

-0.26 

97.74 

16 

31 

0.071 

-2.30 

-1.84 

96.16 

17 

33 

0.517 

-1.15 

-0.92 

97.08 

18 

.35 

0.911 

-0.43 

-0.34 

97.66 

19 

37 

0.039 

-2.55 

-2.04 

95.96 

20 

39 

0.586 

-1.03 

-0.82 

97.18 

.21 

41 

0.869 

-0.53 

-0.42 

97.58 

22 

43 

0.017 

-2.85 

-2.28 

95.72 

23 

45 

0.640 

-0.93 

-0.74 

97.46 

24 

47 

0.824 

-0.62 

-0.50 

97.50 

25 

49 

0.005 

-3.26 

-2.61 

95.39 

26 

51 

0.706 

-0.84 

-0.67 

97.33 

27 

53 

0.776 

-0.71 

-0.57 

97.43 

28 

55 

0.001 

-4.22 

-3.38 

94.62 

29 

57 

0.756 

-0.75 

-0.60 

97.40 

7»3S»h^3I'  a»Mmus 


The  third  example  is  to  duplicate  Ref*  10  for  a  curve  of  experimental 
points  that  are  less  regular  than  Refs.  1-4  in  the  periodic  portrayal. 

The  smooth  curve  of  Ref.  10  could  not  be  fitted  to  an  exponential  or  power 
equation.  However,  for  F(X)  the  point  by  point  estimated  values  for  each  equal 
increment  of  t  ■  200  sec  are  listed  in  Table  III.  The  first  point  was 
at  200  seconds  (the  second  experimental  point)  which  was  selected  for 
N}*  The  parameters  an<*  °E  were  heuristlcally  selected.  The 

constants  and  calculated  values  are  listed  in  Table  III,  and  the  points 
are  shown  in  Fig.  3(b)  by  the  solid  dots.  The  polarity  of  ths  17th 
through  29th  data  points  were  reversed  from  the  expected  polarity  to 
exhibit  agreement  with  the  recoautructed  experimental  values.  The  open 
circles  are  the  experimental  values  as  estimated,  from  Ref .  10  with  which  ths 
predictive  points  are  r;<>t  in  agreement.  The  positions  of  ths  periodicities 
of  the  two  sets  of  data  are  in  close  agreement  with  the  exceptions  at  the  8th, 
20thand  25th  point.*.  The  values  of  most  of  the  data  points,  21  of  ths  29, 
are  nearly  identical,  and  with  the  exception  of  predictive  point  No.  8, 
the  other  seven  are  in  reasonably  close  agreement. 

IV.  DISCUSSION.  The  subject  of  this'  paper  encounters  a  paradox  In  the 
distribution  of  the  measured  values  of  a  physical  parameter  of  an  experiment.  The 
binomial  distribution  is  a  mathematical  treatment  of  random  events  whose 
measured  values  are  discrete,  whereas  the  Caussian  distribution  is  a  mathe¬ 
matical  treatment  of  random  events  whose  measured  values  art  not  discrete 
and  cannot  be  exactly  duplicated.  The  measured  values  of  a  measurement 
are  considered  to  be  Independent  of  the  sequence  of  the  measurements  and  the 
values  randomly  distributed  about  some  mean.  The  independence  and  randomness 
of  the  measured  values  from  event  to  event  "vary  in  an  irregular  manner  that 
defies  all  attempts  at  prediction"  (12).  However,  since  it  was  pointed 
out  in  this  paper  that  most  plots  of  data  points  are  portrayed  in  some 
periodic  manner,  the  data  points,  in  principle,  should  be  predictive. 

The  predlctivene8s  was  demonstrated  in  this  paper,  albeit  ex  post  facto. 

The  data  points  for  three  experiments  were  closely  duplicated  after 
heuristlcally  determining  three  constants*  It  must  be  stated  that  the 
mean  (tbe  equation  of  the  experiment)  was  known.  Thus,  in  this  context, 
the  measured  data  points  are  predictive,  hence  the  paradox. 

The  Gaussian  distribution  is  one  (of  two  primary  concepts)  equation  used 
in  this  paper.  The  treatment  of  the  equation  to  obtain  the  comparative 
probability  eliminated  the  dependence  of  the  probability  from  all  parameters 
except  one:  The  arithmetic  deviation  coefficient.  Solving  for  the  co¬ 
efficient,  it  was  found  that  the  coefficient  was  a  function  of  the  probability 
which  was  identified  as  a  form  of  the  self-information  equation.  The  signi¬ 
ficance  of  the  relationship  between  the  coefficient  and  tha  information 
equation  cannot  be  developed  at  this  time,  but  that  the  Gaussian  distribution 
aquation  is  a  form  of  the  information  aquation  can  be  recognised. 

The  second  concept  is  the  heuristic  equation  relating  the  probability  of 
a  measured  value  as  u  function  of  sequential  measurement.  If  a  well  ordered 
experiment  is  performed  such  that  upon  equal  increments  of  an  Independent 
parameter,  the  measured  value  is  recorded  in  the  identical  manner,  Including 
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Fig.  3.  Duplications  of  (a)  the  cusp  portrayal  and  (b)  the  cusp-sinusoidal 
portrayal  of  experimental  data  points. 


Table  III.  Constants  and  "periodic”  data  points  generated  ae  a 
function  of  measurement  for  the  cusp-sinusoidal  example  for  Ref.  9. 

F(X)  -  R(t) 

0.00,  a2  ■  2.00,oe  »  0.3 


N 

kN 

ktfE 

R(t) 

R(t)+kj|OE 

1 

0 

.1.000 

0.000 

0.00 

66.2 

66.2 

2 

2 

0.103 

+2.13 

+0.64 

59.0 

59.6 

3 

4 

0.517 

+1.15 

+0.35 

53.7 

54.1 

4 

6 

0.888 

-0.49 

-0.15 

49.3 

49.1 

5 

8 

0.016 

-2.87 

-0.86 

46.6 

45.7 

6 

10 

0.676 

-0.89 

-0.27 

•  44.2 

43.9 

7 

12 

0.783 

+0.70 

+0.21 

42.0 

42.2 

8 

14 

0.000001 

,+5.20 

+1.56 

40.3 

41.9 

9 

16 

0.655 

+0.92 

+0.28 

38.7 

39.0 

10 

18 

0.688 

-0.86 

-0.26 

37.4 

37.1 

11 

20 

0.010 

-3.03 

-0.91 

36.2 

35.3 

12 

22 

0.854 

-0.56 

-0.17 

35.1 

34.9 

13 

24 

0.601 

+1.01 

+0.30, 

34.2 

34.5 

14 

26 

0.035 

+2.59 

+0.78 

33.4 

34.2 

15 

28 

0.908 

+0.44 

+0.13 

32.6 

32.7 

16 

30 

0.521 

-1.14 

-0.34 

31.9 

31.6 

17 

32 

0.069 

+2.31 

+0.69 

31.3 

'  32.0 

18 

34 

0.947 

+0.33 

+0.10 

30.7 

30.8 

19 

36 

0.448 

-1.27 

-0.50 

30.2 

29.7 

20 

38 

0.109 

-2.10 

-0.63 

29.9 

29.3 

21 

40 

0.973 

-0.23 

-0.07 

29.4 

29.3 

22 

,  42 

0.382 

+1.39 

+0.42 

29.0 

29.4 

23 

44 

0.153 

+1.94 

+0.58 

28.7 

29.3 

24 

46 

0.990 

+0.14 

+0.04 

28.4 

23.4 

25 

48 

0.322 

-1.51 

-0.45 

28.2 

27.7 

26 

50 

0.200 

-1.79 

-0.54 

28.0 

27.5 

27 

52 

C.998 

+0.06 

+0.02 

27.8 

27.8 

28 

54 

0.269 

+1.62  • 

+0.49 

27.6 

28.1 

29 

56 

0.248 

+1.67 

+0.50 

27.4 

27.9 
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time  lapses,  the  data  points  appear  to  be  portrayed  in  a  periodic  fashion  in 
about  either  every  3-4  or  every  10-12  measurements.  The  probability  of  the 
measured  value  Is  a  function  of  the  measurements  for  -3.00  <  k«KN  <  +3.00 
and  follows  a  cosine-squared  law  that  nearly  duplicates  the  Gaussian  distri¬ 
bution  probability.  The  probability  dependence  on  measurements  cannot  be 
derived  from  first  principles  but  is  heurlstically  justified  by  the  fact  it 
enables  the  "predictive"  values  to  agree  with  the  experimental  data. 

The  duplication  of  the  experimental  data  points  that  ere  portrayed  in  a, 
sinusoidal  fashion  is  in  Itself  not  too  profound.  In  fact,  for  Refs.  1-4, 
the  points  can  be  readily  duplicated  by  nearly  any  periodic  mathematical 
treatment.  However,  the  heuristic  method,  when  applied  to  the  duplication 
of  the  points  that  are  portrayed  in  cusps  (as  in  Refs.  6-9)  and  combination 
of  cusps  and  sinusoids!  periodicities  (as  in  Ref.  10),  the  method  must  be 
considered  as  being  significant.  The  significance  is  emphasised  when  form  e 
cursory  examination  some  of  the  data  points  appear  to  be  random  but  are 
duplicated  by  the  method.  Obviously,  all  of  the  published  data  points 
cannot  be  duplicated  by  the  limited  treatment  of  the  heuristic  method 
described  in  this  papnr. 

It  is  poutulated  that  the  sinusoidal  portrayal  of  data  points  in 
which  the  measured  parameter  does  not  have  a  fixed  bound  or  physical 
barrier.  Examples  of  the  sinusoidal  characteristic  would  be  the  measure 
of  electron  trapping  as  a  function  of  time  (Ref.  1),  and  the  determination 
of  the  beam  displacement  of  s  reflected  electromagnetic  wave  aB  a  function 
of  distance  (Faf.  3). 

It  ia  postulated  that  the  cusp-type  portrayal  of  data  points  is  charac¬ 
teristic  of  an  experiment  in  which  the  measured  parameter  (the  dependent 
variable)  has  a  fixed  bound  or  physical  barrier.  Examples  of  the  cusp 
characteristic  would  be  the  "no  influence"  effect  of  source  strength  as  a 
function  of  time  (Ref.  6),  and  the  measurement  of  the  length  of  a  meter  stick 
by  the  eclipalng  of  the  position  of  a  light  source  by  the  end  of  the  stick. 

There  are  two  baffling  considerations  that  were  required  for  obtaining 
agreement  of  the  points  in  two  of  the  examples.  The  first  is  that  two 
experimental  points  had  to  be  by-passed  in  Ref  7,  and  the  second  is  that  a 
change  of  polarity  for  the  arithmetic  deviation  coefficient  for  Raf.  10. 

These  considerations  may  be  resolved  if  the  constants  of  the  heuristic  equation 
were  evaluated  simultaneously  with  the  experimental  data  points  of  an 
experiment  performed  in  a  human  factors  evaluation  laboratory.  In  such  a 
laboratory,  the  constants  may  be  determined  from  the  nature  of  the  experiment 
and  the  experimental  procedures,  and  thus  in  conjunction  with  the  equation 
governing  the  experiment,  the  measured  data  points  may  be  indeed  predictive. 
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ABSTRACT.  The  numerical  treatment  of  nonlinear  model  fitting 
problems  often  can  be  elmplified  by  manipulating  the  model  equations. 
Algebraic  manipulations,  including  nonlinear  trane format lone  of  model 
parameters,  do  not  change  the  numerical  result  of  the  adjustment. 
Therefore,  such  manipulations  can  be  a  powerful  method  to  improve 
the  performance  of  solution  algorithms.  Nonlinear  transformations  of 
the  observations ,  on  the  other  hand,  do  change  the  numerical  results 
unless  the  normal  equations  are  transformed  accordingly.  The  latter 
transformation  has  been  negleoted  by  previous  authors  and  this  article 
provides  a  complete  set  of  formulas  that  are  needed  to  implement 
transformations  of  observations.  The  transformations  are,  however, 
in  general  less  useful  than  parameter  transformations  but  may  have 
applications  in  particular  situations. 

1 .  INTRODUCTION .  A  mathematical  model  fitting  problem  arises 
when  one  compares  real  observations  with  theoreioal  predictions.  The 
observations  always  aontain  observational  inaccuracies  and,  likewise, 
the  theory  of  the  prediction  can  be  inadequate.  If  discrepancies 
between  observations  and  predictions  are  unacceptably  large  for  a 
particular  situation  then  one  is  faced  with  the  task  to  adjust  in  a 
rational  manner  either  the  observations,  or  the  theory,  or  both  so 
that  an  acceptable  mathematical  description  of  the  event  can  be 
established.  The  problem  can  be  subdivided  conveniently  into  three 
aubtaoks,  each  of  which  requires  a  different  approach  and  background 
information. 

First,  one  has  to  chose  a  modal.  Normally,  this  requires 
supporting  information  from  engineering,  physics,  geometry,  etc., 
which  may  suggest  or  postulate  a  reasonable  mathematical  description 
of  the  observable  event.  We  shall  assume  in  this  article  that  the 
model  is  formulated  as  a  system  of  equations  containing  observations 
and,  possibly,  also  some  undetermined  model  parameters. 

Once  the  model  is  selected,  one  can  compare  predicted  values 
of  observable  quantities  with  corresponding  observations.  The 
comparison  provides  the  basis  for  a  rational  adjustment  of  the 
observations  and/or  of  the  model.  This  subtask  of  the  problem  is  a 
purely  mathematical  part  of  model  fitting  and  it  belongs  to  the 
category  of  ill-posed  problems.  Its  mathpmatical/numerical  treatment 
is  independent  of  the  other  two  subtaBks,  i.e.,  of  applications.  We 
shall  be  concerned  with  this  part  of  the  problem  in  the  present  article. 
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After  the  adjustments  have  been  carried  out,  one  has  to  validate 
the  mathematical  model,  unless  it  has  been  prescribed,  e.g.,  by  the 
geometry  of  the  event.  The  validation  involves  typically,  but  not 
necessarily,  a  statistical  analysis  of  the  discrepancies  between  ob¬ 
servations  and  predictions.  The  result  of  the  validation  process  may 
bo  a  new  formulation  of  model  equations  and  subsequent  fitting,  i.e., 
a  repetition  of  the  whole  task  until  some  validation  criterion  J.b 
satisified.  We  shall  not  discuss  this  part  of  the  problem,  noticing 
only  that  the  results  of  the  second  aubtask  provide  the  data  basis 
necessary  for  a  validation. 

If  the  model  equations  are  not  linear  then  the  model  fitting 
problem  generally  leads  to  systems  of  complicated  simultaneous  equations 
and  corresponding  numerical  difficulties  may  arise.  Often  the  numerical 
treatment  can  be  simplified  by  a  reformulation  of  the  model  equations, 
particularly  by  Introduction  of  new  variables  through  variable  trans¬ 
formations.  Such  manipulations  have  been  suggested  in  textbooks*-?  and 
are  routinaly  used  in  applications.  Examples  of  recently  published 
applications  where  variable  transformations  have  been  used  are  refer¬ 
ences  8,  9,  and  10. 

A  closer  investigation  of  variable  transformations  in  model  fitting 
problems  suggests  that  the  formulations  should  be  used  more  cautiously 
than  some  of  the  texts  suggest.  Therefore,  we  shall  present  in  this 
article  an  investigation  of  some  consequences  of  the  transformations 
and  draw  conclusions  about  their  usefulness  for  the  simplification  of 
the  numerical  treatment  of  model  fitting  problems, 

in  Section  2  we  shall  formulate  the  mathematical  model  fitting 
problem  in  general  terms  and  discuss  the  effeats  that  can  ba  anticipated 
from  manipulatione  of  model  equations.  In  Section  3  we  shall  specialise 
the  considerations  to  nonlinear  least  squares  problems  and  produae 
explicit  formulas  that  are  needed  in  such  problems.  Some  examples  will 
be  presented  in  Section  4,  and  Section  5  will  summarise  tha  conclusions 
that  can  be  drawn  from  the  theoretical  discussions  and  from  examples, 

2.  GENERAL  ASPECTS  OF  MATHEMATICAL  MODEL  FITTING.  Let  the  model 
equations  be 

A(X)6  -  0,  12.1) 

where  XeRn  iB  the  vector  of  all  observations,  BtvP  is  a  model  parameter 
veator,  and  A(X)  ia  an  operator  that  operates  on  6  and  has  a  range 
Rr.  We  assume  that  the  following  relations  hold  between  the  dimensions 
n,  r,  and  pi 

n  >  t  >  p  >  0,  (2.2) 
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By  permitting  the  dimension  p  to  be  Eero,  we  include  in  our  consider¬ 
ations  also  cases  in  which  the  model  equations  do  not  oontain  free 
parameters.  Then  equation  (2.1)  reduces  to  A(X}«0. 

Typical  for  applications  are  cases  in  which  the  r  equation  (2.1) 
for  6  are  independent  and,  because  of  (2.2),  do  not  have  a  solution. 

Than  one  replaces  the  model  equations  by  another  system 

A(X)G  «  0,  (2.3) 

chosing  the  operator  A(X)  such  that  it  approximates  MX)  and  has  a 
solution.  The  determination  of  A(X)  can  be  considered  as  the  central 
part  of  the  model  fitting  problem. 

in  order  to  have  a  measure  for  the  approximation  we  introduce 
a  metric  for  the  operators.  Let  p[A(X),  MX) 1  be  a  metric.  Then  one 
can  formulate  the  mathematical  model  fitting  as  the  following  con¬ 
strained  minimization  problem) 

A(X)e  -  0  ,  W{p[A(X),  A(X)  ]  -  mir.. ,  (2.4) 

where  W{p}  is  generally  a  convex  object  function.  The  choice  of  the 
metric  p  and  of  the  object  function  Wip)  determines  the  type  of  the 
model  fitting,  e.g.,  least  squares,  maximum  norm,  etc. 

we  shall  now  discuss  the  selection  of  an  approximate  operator 
A(X) .  First,  we  notice  that  the  model  operators  MX)  and  A(X)  are 
generally  needed  and  defined  oi.'y  within  a  finite  neighborhood  of  the 
observations  X.  Therefore,  assumptions  about  properties  of  the  operators 
need  to  be  made  for  that  neighborhood  only.  Let  the  neighborhood 
consist  of  all  points  Z  **  x+C,  whereby  C  is  restricted  oomponant-wise 
by 

^  Ci  <  ,  i  ■  1,  2,  . . . ,  n.  (2.5) 

The  Intervals  (y. ,  I\)  normally  contain  zero,  but  exceptions  are 
possible  and  do  occur  in  applications.  Second,  we  assume  that  within 
the  neighborhood  (2.5)^  A(Z)  is  a  continuous  function  of  Z.  Then  a 
reasonable  choice  of  A(X)  is 

A(X)  -  A (X+C) .  (2.6) 

The  choice  achieves  a  natural  parametrization^of  the  approximation. 

The  approximation  parameter  is  the  vector  CtK  and  the  operator  A(X) 
depends  continuously  on  the  parameter  within  the  restrictions  (2.5). 

The  parametrized  model  fitting  problem  can  be  formulated  as 
follows) 
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A (X+C) 6  -  0, 

w{p [A(X+C) ,  A(X) ] }  -  min. 


(2.7) 


The  quantities  to  be  determined  by  equntion  (2.7)  are  the  approximation 
parameter  c  and  the  model  parameter  0.  We  assume  that  the  solution 
vector  C  is  within  the  limits  specified  by  equation  (2.5). 


We  will  need  in  the  sequel  some  differentiability  properties  for 
the  modal  operator.  As  far  as  x  is  concerned,  we  assume  the  properties 
to  hold  within  the  neighborhood  (2.5).  With  respect  to  6  ws  assume 
that  a  similar  neighborhood  exists  in  the  vicinity  of  the  solution  of 
equation  (2.7)  ir.  which  A(X)6  Is  a  continuous  function  of  0.  fho 
differentiability  assumptions  rra  that  A(X+C)6  is  twice  differentiable 
with  respect  to  all  its  n+p  arguments  within  aha  cartesian  product 
rpace  of  the  neighborhoods  of  x  and  9.  We  also  assume  that  within 
that  space 


rink 


3A 

3X 


r, 


(2.8) 


and  define 

p[A(Z),  A (X) )  -  | |B  -  X| | . 


(2.9) 


p  is  a  metric  within  the  neighborhood  in  which  (2.8)  holds.  We  also 
assume  that  the  model  equations  do  not  contain  redundant  parameters. 
The  assumption  may  be  expressed  as  the  requirement 


rank 


3A(X)9 

39 


P. 


(2.10) 


with  the  specialization  (2.9),  the  model  fitting  problem  becomes 


A ( X+C ) 8  -  0, 

W{p[A(X+C),  A (X)  ]  }  -  W{  |  |c |  j  }  «*  min. 


(2.11) 


Equation  (2.11)  is  an  abstract  formulation  of  common  modal  fitting 
problems.  The  difference  C  between  the  observations  X  and  tho 
"corrected  observations"  X+c  is  called  the  residual  vector.  In  the 
formulation  (2.11)  we  require  that  a  norm  of  the  residual  vector  be 
minimized,  subject  to  model  equations  which  have  to  be  satisfied  at 
X+C.  The  model  parameter  vector  0  is  not  essential  in  this  formulation 
The  number  of  model  parameters  may  be  zero  and  it  is  normally  orders 
of  magnitudes  smaller  than  the  number  of  approximation  parameters,  i.e. 
residuals.  The  determination  of  0  aan  oe,  of  course,  in  some  applica- 
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tions  more  important  than  the  determination  of  C,  but  this  ia  not 
alwaya  the  oaae. 


A  least  aquarea  model  fitting  problem  1b  a  special  case  of  (2.11) , 
characterized  by  a  particular  choiae  of  the  norm  in  the  definition 
(2.5),  and  of  the  object  function  W{p}.  The  least  squares  metric  ia 


p[A(Z),  A(X)1  -  ||Z-X|1  -  ((Z-X)Vl(Z-X)]*  , 


(2.12) 


where  P  is  an  eatimate  of  the  variance-covariance  matrix  of  the 
observations.  The  least  square*  object  function  ia 

W{p)  -  p2  .  (2.13) 

Therefore,  the  least  squares  model  fitting  problem  is  definsd  by 

A(Xtc)t  «  0, 


w  » 


i  m2  1' 

o  ■  c  R  c  *  min . 


(a.ui 


In  equation  (2.14)  we  have  used  c  and  t  instead  of  C  and  0,  respectively, 
thu«  indicating  the  least  squares  values  of  both  parameter  vectors. 

The  use  of  R ”1  as  a  norm  matrix  in  the  definition  (2.11)  makes 
the  norm  ||c||  and  W  dimensionless,  which  is  very  convenient  when 
fitting  results  are  compered.  If  the  varianct-oovarianoe  matrix  R 
is  known  exaatly,  than  ths  volution  of  aquation  (2.14)  ia  a  maximum 
likelihood  solution  of  the  approximation  problem1'1-.  The  seme 
maximum  likelihood  solution  is  obtainsd  if  R  approximates  ths  variance- 
covariance  matrix  up  to  an  unknown  factor.  In  applications  ona  has 
to  be  content  with  an  estimate  of  R.  Then  often  the  off-diagonal 
elements  are  assumed  to  be  zero  az  e  mattsr-of-oourse.  Beoauss  the 
results  of  the  model  fitting  depend  on  K,  such  assumptions  should 
not  be  made  vdthout  having  reasons  that  zero  is  a  better  approximation 
than  a  non- zero  value.  The  theoretical  treatment  is  not  complicated 
by  the  assumption  that  R  is  not  diagonal,  nor  are  the  numerical 
complications  unaurmountsbls .  Realistic  estimates  of  R  are,  however, 
important  for  ths  interpretation  of  ths  results,  and  for  ths  validation 
of  the  fitting. 

We  solve  the  optimization  problem  (2.11)  or  (2.14)  using  Lagrange 
multiplier  technique,  and  call  the  multipliers  correlates,  as  usual 
in  adjustment  problems.  Let  KeRr  be  a  correlate  vector  and  let  the 
modified  objeat  function  be 


W  »  ~  W{| |c||}  -  KTA(X+C)6  . 


(2.15) 
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Necessary  conditions  for  tha  solution  of 
are  obtained  by  setting  saro  the  partial 
to  the  unknown  C,  6,  and  K.  This  yields 
equations. 


the  optimisation  problem 
derivatives  of  H  with  respect 
the  following  set  of  normal 


jfc  I  |c|  l  >  -  |~  (KTA(X+C)0]  -  0  , 


(2.16a) 


~  IKTA(X4C)6)  -  0, 


(2.16b) 


A(X+C)0  -  0.  (2.160) 

The  solution  of  tha  modal  fitting  problem  (2.11)  is  among  tha  solutions 
of  equations  (2.16).  On  the  other  hand,  one  oannot  guarantee  that  a 
particular  solution  of  tha  normal  aquations  corresponds  to  tha 
absolute  minimum  solution  of  equation  (2.11)#  nor  is  the  uniqueness 
of  the  solution  given.  An  investigation  of  these  complications  is  not 
tha  subject  of  this  paper.  Mostly,  such  problems  can  be,  and  are 
taken  care  of  by  ad  hoc  measures  based  on  background  information  from 
tha  application.  Therefore,  wa  simplify  our  prssant  theoretical  dis¬ 
cussion  by  assuming  in  this  section  that  a  numerical  solution  of 
equations  (2.16)  can  be  obtained,  and  that  it  has  bean  verified  as  the 
absolute  minimum  solution  of  equation  (2.11). 

In  least  squares  problems,  tha  first  term  3W/3C  in  equation  (2.16a) 
is  linear  with  respect  to  C.  Nonlinear  expressions  which  could  be 
possibly  simplified  by  algebraic  manipulations  may  occur  in  the  second 
term  in  aquation  (2.16a),  and  in  aquations  (2.16b)  and  (2.16c).  Tha 
structure  of  these  terms  strongly  depend  on  the  form  in  which  tha  modal 
equations  (2.16c)  era  cast,  and  it  is  obvious  that  simplifications  can 
be  achieved  by  proper  formulations.  Particularly,  one  does  not  have 
to  insist  that  each  modal  aquation  be  solved  for  a  "dependant"  observa¬ 
tion.  Such  a  form  is  assumed  in  most  textbooks  on  data  reduction  and 
postulated  in  computer  programs  for  data  reduction  problems.  Quite 
often  an  implicit  formulation  of  the  aquations  (2.16c)  can  be  simpler, 
producing  also  simpler  expressions  for  the  derivatives  in  equations 
(2.16a)  and  (2.16b).  Tha  solution  of  tha  problam  (2.11)  is,  of  course, 
independent  of  the  particular  form  in  which  tha  model  equations  are 
cast.  This  remark  is  trivial  in  the  present  context,  and  it  is  a 
consaquence  of  the  formulation  of  the  model  fitting  problem  by  equation 
(2.11).  Reference  12  reports  about  numerous  unsuccessful  attempts  to 
achieve  a  similar  invariance  statement  when  tha  problem  was  formulated 
differently. 

The  aforementioned  manipulations  of  the  modal  operator  A(X)0  can 
also  include  nonlinear  transformations  of  the  parameter  6.  Such 
transformations  do  not  affect  the  definition  of  the  metric  p,  because 
the  metric  of  the  operator  Is  independent  of  the  operand.  Therefore, 
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the  transformations  do  not  affaot  ths  first  term  in  aquation  (2.16a) 
sithar  and  ars  a  powarful  tool  for  tho  simplification  of  tha  rast  of 
tha  aquations.  An  axampla  in  which  nonlinaar  paramatar  transformations 
ara  usad  to  linaariaa  tha  modal  aquations  is  raportad  in  rafaranoa  9. 

In  Saotion  4  wa  shall  give  othar  axaaplas. 

Tha  formal  proaadura  of  replacing  paramatars  is  as  follows i 
Suppose  that  one  wants  to  raplaaa  tha  paramatar  6  by  o  whereby  both 
paramatars  are  related  by  a  nonsingular  function 

e«w(c).  (2.17) 

(Regularity  of  tha  transformation  naad  to  be  assumed  only  within  a 
neighborhood  of  tha  solution.)  Let  tha  modal  aquations  ba  in  terms 
of  a 


A (X) 0  -  0.  (2.18) 

The  operator  A  can  ba  obtained  from  A  always  by  tha  definition 

A(x)o  a  A(X)w(o) ,  (2.19) 


however,  often  one  can  find  othar  equivalent  formulations  that  ara 
simpler.  Tha  metric  jT  associated  with  A  is  defined  as  in  aquation 
(2.9) 


P  [A(Z),  A(X)J  - 

IIMI- 

(2.20) 

with  this  definition  and  tha  same  object  function  W{p} 
obtains  tha  normal  aquations 

as  before  one 

|  «  tllcll)  - 

*  3C 

—  [ic^AtX+CJa]  -  o, 

3C 

(2.21a) 

~  [K^MX+Cjo]  - 

0, 

(2.21b) 

A(X+C) 0  -  0. 

(2. 21n) 

The  solution  vectors  of  aquations  (2.16)  and  aquations  (2.21)  ara 
related  by 


C  -  C  ,  6  -  w(o) 


(2.22) 


13) 


The  vector*  K  and  K  can  be  computed  from  these  value*  using  formula* 
given.  In  the  next  section. 

The  relation  (2.22)  i*  again  a  single  consequence  of  the  formula¬ 
tion  (2.11)  of  the  model  fitting  problem.  Bender?  prove*  the 
correspondence  (2.22)  for  a  particular  transformation  and  application , 
and  indicates  that  previous  developers  of  software  for  such  problems 
were  not  aware  of  the  relation. 

If  the  solution  of  the  model  fitting  task  has  bean  found  from 
equation  (2.21)  in  terms  of  er,  but  the  parameter  vector  9  is  of 
interest ,  then  one  needs  in  addition  to  aquation  (2.22)  another  formula 
for  the  accuracy  of  9.  Lot  us  assume  that  the  solution  algorithm  for 
equation  (2.21)  has  also  provided  information  about  the  accuracy  of 
o  in  form  of  an  estimate  V0  of  the  variance-covariance  matrix  of  the 
components  of  o.  (In  Section  3  we  shall  give  formulas  for  Va  in  least 
squares  problems.)  Then  an  estimate  of  the  variance-covariance  matrix 
Vq  of  the  components  6f  9  can  be  obtained  by  applying  the  linearised 
law  of  variance  propagation  to  the  relation  (2.22).  The  result  is 


v  -  S3L  v  (2SL)t 

v9  3o  vo  W 


(2.23) 


More  complicated  are  consequences  of  such  manipulations  of  the 
modal  equations  that  involve  transformations  of  the  observations.  This 
is  so  because  the  transformations  now  affect  the  definition  of  the 
norm  p.  Next,  we  shall  consider  such  transformations. 

Let  a  transformation  of  observations  be 


Y  -  v(X)  (2.24) 

with  the  inverse 
X  ■  u(Y)  . 


We  assume  that  the  transformation  is  regular  within  the  neighborhood 
(2.5),  including  the  solution  X+c,  and  that  the  function  u(Y)  is 
there  twice  differentiable.  The  model  equations  (2.1)  are  replaced 
by  equivalent  (usually  simpler)  equations 


A(Y)9  -  0.  (2.25) 


The  operator  A(Y)9  can  be  obtained,  e.g.,  by  the  definition 
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A(Y)6  a  A (u(Y) ) 0 i 


(2i36) 


but i  as  in  ths  oasa  of  parameter  transformational  usually  othar 
aquivalant  formulations  can  ba  found  that  ara  simpler. 

Whan  wa  formulata  tha  modal  fitting  problam  in  tarns  of  Y,  wa 
hava  to  kaap  in  mind  that  tha  goal  is  to  mininisa  tha  distanca  c 
batwaan  tha  actual  obsarvatlona  X  and  thair  oorraotad  values  x+c. 
in  laast  squaraa  problama,  only  such  a  minimisation  yialds  undar 
conditions  a  maximum  likalihood  solution.  Than  tha  minimisation 
problam  (2.11)  is 


Y  -  v(X) i 
A (Y+B) 0  -  0, 

W(f |u(Y+B)  -  x| j }  -  min. 

Tha  normal  aquations  for  tha  problam  (2.27)  ara 
W{  |  |u(Y+B)-x|  |  >  -  Ij  [KTA(Y+B)8]  -  0, 
[KTA(Y+B)9]  -  0, 

A (Y+B) 9  -  0. 


(2.27) 


(2.28a) 

(2.28b) 

(2.28o) 


Tha  first  tarm  in  aquation  (2.28a)  is  not  linaar  with  respect  to  tha 
unknown  B  unlasa  tha  transformation  (2.24)  is  lines;.  Tharefora«  a 
nonlinaar  transformation  that  produces  an  operator  A(Y)6  which  is 
simpler  than  tha  original  operator  A(X)6,  introduces  nonlinear  tarma 
in  aquation  (2.28a).  Tha  new  nonlinaaritias  may  offset  tha  advantages 
gained  by  a  simplification  of  tha  othar  terms  in  tha  aquations. 

Wa  shall  pursue  this  point  further  in  tha  next  section  and  show 
in  detail  how  tha  normal  aquations  and  algorithms  ara  affected  by 
transformations  of  observations  specifically  in  least  squares  problama 

3.  LEAST  SQUARES  MODEL  FITTING.  Wa  consider  in  this  section 
tha  effects  of  variable  transformations  on  laast  squares  modal  fitting 
problems.  Wa  shall  first  derive  the  basic  aquations  for  nonlinaar 
laast  squares  problems  in  terms  of  tha  original  observational  and  then 
show  how  the  equations  are  affected  by  a  transformation  of  tha 
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observations,  ws  simplify  our  notation  by  defining  a  vector  function 
F(x,0)  by 

F(X,9)  a  MX)  0.  (3.1) 

Than  the  model  equation  (2.1)  is 

F(X,0)  ■  0,  (3.2) 

and  the  least  squares  model  fitting  problem  (2.14)  is 
F(X+c,t)  -  0, 

| | a | | 2  *  cTr"^o  ■  min.  (3.3) 

In  the  sequel  we  will  use  subscripts  to  denote  derivatives. 

Also,  because  derivatives  of  F(X+e,t)  with  respect  to  o  are  identical 
to  derivatives  with  respect  to  X  we  shall  use  the  subscript  x  for  both. 
Thus,  a.g., 

Fx<X+c,t)  -  “F(X-«J,t)  a  F(X+o,t) 
and 

2  2 

[K^FtX+O.tJ^  -  [KTF(X+C,t)l  a  [KTF(X+C,t)] 

are  matrices  with  the  dimensions  ran  and  nxp,  respectively. 

Using  this  notation,  the  normal  equations  corresponding  to  the 
problem  (3.3)  are 


r"1c  -  kTrx(x+c,t)  -  o, 

(3.4a) 

kTFt(X+o,t)  -  0, 

(3.4b) 

F (X+c ,t)  “  0. 

(3.4c) 

134 


»»iwihWiw 


The  normal  aquation*  ara  in  ganaral  nonlinear  with  respect  to  a 
and  t.  Therefore,  thalr  numarloal  solution  will  raquira  soma  kind  of 
iteration.  Ha  obtain  second  order  iteration  aquation*  for  equations 
(3.4)  by  expanding  the  normal  equations  at  an  approximation  to  the 
solution  and  keeping  the  linear  terms  of  the  axpanoion.  Let  the 
approximation  to  tha  solution  be  C,  K,  and  T,  and  that  the  corresponding 
oorraotiona  be  c,  k,  and  t.  Than  tha  expansion  yields  tha  following 
Newton  aquations  for  the  corrections t 


U-R(KTr)xx)C  -  RFxMK+k)  -  H(KTP)xtT-  -  c, 


(3.5a) 


(KTP)  txc  +rJ'(K+tc)  +  <KTF)ttt 


(3.5b) 


+  FfcT  «  -  P. 


(3. So) 


The  arguments  of  V  and  its  derivatives  in  equations  (3.5)  are  X+C  and  T. 

Newton-Raphson  iteration  equations  can  be  established  by  suitable 
manipulations  of  equations  (3.5)®»13'14' 15 .  X  eat  of  such  iteration 
equations  are  given  in  tha  Appendix.  Most  authors  simplify  aquations 
(3.5)  by  neglecting  oil  terms  that  contain  second  order  derivatives1/31*'1®'37. 
This  yields  so-called  Gauss-Newton  procedures  that  have  theoretically 
only  linear  convergence  and  that  also  may  have  other  peculiarities13. 


The  final  step  in  a  model  fitting  problem  is  to  obtain  variance 
estimates  of  the  solution  in  terms  of  the  estimated  variances  of  the 
observations.  He  shall  restrict  ourselves  in  this  artiole  to  the 
estimation  of  the  accuracies  of  the  least  squares  value  t  of  the 
parameter  vector,  and  show  how  the  estimation  formulas  change  dun  to 
transformations  of  observables.  We  shall  us*  the  linearised  variance 
propagation  formula  for  the  estimates.  Estimates  of  the  accuracies 
of  the  corrected  observations  x*X+c  can  be  obtained  by  analogous 
processes . 


The  formulae  can  be  derived  from  the  linear  terms  of  an  expansion 
of  the  normal  equations  (3.5)  at  the  solution13.  Let  dx,  dk.  and  dt 
be  the  differentials  of  the  solution  vectors  x-X/c,  k  and  t, 
respectively.  Then  the  expansion  yields 


(l-R(kTF)KX]dx  -  WxTdk  -  K(kTF)xtdt  -  dX, 


(3.8a) 


(kTF)tx'2x  +  pjjdk  +  (kTF)ttdt  -  0r 


(3.6b) 


+  Ftdt  -  0. 


(3.6o) 


IMUIfrWfcJiMMSiU.' 
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The  arguments  of  F  and  ita  derivatives  in  equations  (3.6)  are  x  and  t. 

By  manipulations  of  aquations  (3.6)  that  can  be  done  in  various 
ways13#1-8  one  obtains  linear  relations  between  dt  and  dx,  and 
between  dx  and  dx,  respectively.  Let  the  former  relation  be 


N  dt  -  a  dx.  (3.7) 


(Explicit  formulas  for  N  and  S  are  given  in  the  Appendix.)  Then  the 
estimated  varianae-oovarianoe  matrix  of  the  parameter  veotor  t  is 


It  is  obvious  from  the  derivation  of  equation  (3.8)  that  which 
itself  is  only  a  linearised  approximation  depends  on  seoond  order 
derivatives  of  F.  (The  formulas  In  the  Appendix  show  explicitly  this 
dependency.)  Neglect  of  the  seoond  order  derivative  terms  renders  a 
formula  that  is  theoretically  less  than  first  order  accurate.  Therefore, 
such  a  neglect  has  to  be  justified  in  each  application  by  providing 
estimates  of  the  magnitudes  of  the  negleated  terms.  Of  the  cited 
references,  only  in  references  13,  14,  15,  and  IS  complete  first  order 
formulas  are  used. 

Next,  we  introduce  variable  transformations  into  the  least  squares 
model  fitting  problem.  We  can  restrict  ourselves  to  transformations 
of  observations  because,  ap  shown  in  Section  2,  transformations  of 
model  parameters  have  the  same  effects  as  simple  algebraic  manipulations 
of  the  model  equations. 

Let,  as  in  Section  2,  the  transformation  be  given  by 


V  -  v(X)  (3.9) 

with  the  inverse 
X  -  u(Y) , 

In  terms  of  Y,  the  least  squares  model  fitting  problem  is  defined  by 

Y-v(X),  (3.10a) 

II(Y+b,t)  -  0,  (3.10b) 
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|  |u(Y+b)-x|  I3  -  tu(Y+b)“XjTlf1  [u (Y+b) -X]  -  min.  (3.10c> 

Equation  (3.10b)  is  a  modal  aquation ,  aquivalant  to  aquation  (3.2) 
and  axprasaad  in  terms  of  Y. 

The  normal  aquations  for  the  problem  (3.10)  are 
(uy(Y+B)]V’1  [u(Y+b)-X]  -  kTHy(Y+b,t)  ■  0  (3.11a) 

itTHt(Y+b,t)  -  0,  (3.11b) 

H(Y+b,t)  -  0.  (3,11c) 


Corresponding  Newton  equations  for  corrections  R ,  < ,  end  t  of  approxi¬ 
mate  solutions  B,  Xi  and  T,  respectively,  are 

[I-QS]0-QHy* (K+K)  -  Q(KTH)ytT  -  -  A,  (3.12a) 

(KTH)tya  4-  H**(K+K)  +  (K^H) tt*r  -  0,  (3.12b) 

Hy0  +  HtT  ■  -  H,  (3.12c) 

where 

Q  -  V*VXT  "  (V~1r  ^y1'1  » 

A  -  vx*  [u(Y+B)-X]  ■  Vx.C  -  (u 

H  -  (K*H)yy  -  <uVlC)yy  . 

The  arguments  off  the  functions  H  and  u  in  equations  (3.12)  through 
(3.15)  are  Y+B  and  T,  and  the  last  term  in  equation  (3.15)  is  differ¬ 
entiated  assuming  C-u(Y+B)-X  to  be  constant.  The  term  is  a  symmetric 
nxn  matrix  containing  second  order  derivatives  of  the  transformation 
function  u(Y). 


(3.13) 

(3.14) 

(3.15) 
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A  comparison  of  aquations  (3.12)  with  equations  (3.5)  shows  that 
the  important  changes  in  the  Newton  equations  due  to  the  transformation 
(3.9)  are  in  equations  (3.12a).  The  rest  of  equations  (3.12)  is  formally 
identiaal  to  the  corresponding  terms  in  equations  (3.5),  if  f(x,6)  is 
is  replaced  by  H(Y,0)N.  m  equations  (3.12a)  we  see  three  other  replace¬ 
ments!  the  estimated' yariance-covarianoe  matrix  R  is  replaced  by  Q, 
the  right  hand  side  -C\ls  replaced  by  -h,  and  the  term  (kTf)  is 
replaced  by  S.  ** 

The  replacement  of  R  by  Q  corresponds  to  an  application  of  the 
linearised  variance  propagation  formula  to  the  transformation  (3.9). 

The  replacement  of  the  right  hand  sides  is  a  linearised  transformation, 
of  the  residuals  C  into  the  Y-spaca.  If  the  transformation  (3.9)  is 
linear,  then  only  these  two  replacements  oacur.  If,  however,  the 
transformation  is  nonlinear,  then  the  last  term  in  equation  (3.15) 
does  not  vanish  and,  because  it  contains  second  order  derivatives  of 
u(Y) ,  it  oan  be  quite  complicated.  This  complication  oar.  offset 
algorithmic  advantages  gained  by  a  simplification  of  other  terms  in 
the  Newton  equations. 

Iteration  algorithms  and  formulas  for  the  variances  of  the  solution 
again  can  be  obtained  by  manipulations  of  the  Newton  equations.  Explicit 
formulas  are  given  in  the  Appendix.  We  notice  that  second  order 
Newton-Raphson  algorithms  necessarily  contain  second  order  derivatives 
of  the  modal  function  H  as  well  as  of  the  transformation  function 
u(Y) .  The  coding  of  the  ascend  order  derivatives  oan,  of  course,  be 
avoided  if  first  order  Gauss-Newton  algorithms  are  used.  However, 
variance  estimates  of  the  solution  can  be  calculated  to  a  first  order 
aacuraay  only  if  all  the  second  order  derivatives  are  available. 

The  author  has  carried  out  numerical  experiments  to  determine 
whether  a  solution  of  equations  (3.11)  instead  of  equations  (3.4) 
has  algorithmic  advantages.  The  experiments  were  done  with  the 
utility  programs  described  in  reference  15.  The  programs  permit  one 
to  carry  out  the  calculations  either  in  torms  of  X,  or  in  terms  of  Y, 
and  to  use  either  Newton-Raphson,  or  Gauss-Newton  algorithms.  Tho 
experiments  were  inconclusive,  in  some  examples  the  algorithms  con¬ 
verged  better  when  the  problem  was  formulated  in  X,  in  other  examples 
a  formulation  in  Y-v(X)  produced  better  algorithms.  However,  the 
differences  in  performance  were  never  significant.  This  result  is  in 
strong  contrast  to  similar  experiments  involving  transformations  of 
parameters.  In  thoss  experiments,  a  suitable  parameter  transformation 
often  had  a  dramatic  effaot  on  the  pezformance  of  the  solution  algorithm. 
Soma  examples  are  given  in  the  next  section. 

Another  possible  benefit  from  nonlinear  transformations  of 
observations  could  be  a  simpler  problem  formulation.  The  complexity 
of  the  normal  equations  is  thereby  of.  secondary  Importance,  if  one 
uses  an  available  general  utility  program  for  their  solution.  However, 
the  model  equations  must  be  made  available  to  the  utility  program, 
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which  means  that  tha  aquations  must  ba  programmed.  Than  one  haa  tha 
choice  to  program  either  tha  function  F(x,6)  with  its  first  and  second 
order  derivative*/  or  tha  two  functions  H(Y,8)  and  u(Y)  w'  ih  their 
derivatives.  If  the  transformation  is  nonlinear,  then  normally  the 
programming  of  H  and  u  will  not  be  simpler  than  the  programming  of  T. 

An  exception  may  be  the  situation  where  the  same  transformation  u(Y) 
(e.g.,  polar-cartesian)  is  used  for  several  problems  with  different 
model  functions  H(Y/6)/  so  that  u(Y)  haa  to  be  programmed  only  onoe. 

We  may  conclude  that  in  general  a  transformation  of  observations 
offer  little  or  no  advantages  over  a  formulation  of  the  model  equations 
in  terms  of  the  original  observations.  There  are,  however,  other 
useful  applications  of  such  transformations.  First,  a  graphical 
display  of  tha  results  can  be  clearer  in  terms  of  Y  then  in  terms  of 
X.  Second,  and  more  importantly,  the  transformations  can  be  a  con¬ 
venient  method  to  derive  a  "falsified"  problem  that  oan  be  solved 
easily  and  that  provides  initial  approximations  to  the  unknown  least 
squares  iiolution  vectors.  One  aan  falsify  tha  problem,  e.g.,  by  using 
a  nonlinear  transformation  but  linearising  its  effects  on  the  problem 
formulation.  A  simple  and  effective  falsification  is  to  replace  the 
problem  (3.10)  by 


Y  -  v(X) , 


B (Y+b,t)  -  0, 


b'[uy(Y)R_1uy(Y)]b  -  min. 


(3.16a) 


(3.16b) 


(3.16c) 


The  formulation  is  identical  to  the  correct  formulation  (3.10)  only 
if  the  :r»nsformation  is  linear,  but  the  normal  equations  for  the 
false  pirobLem  (3.16)  are  simple > 


Cf^b  -  K*Hy  (Y+b/t)  -  0, 


K  H  (Y+b,t)  -  0, 


H(Y+b,h)  -  0, 


where 


Q  -  tu^yH^RlUy  (Y)]"1  . 


(3.17a) 


(3.17b) 


(3.17c) 


(3. IB) 


This  system  can  be  much  simpler  and  easier  to  solve  than  equations 
(3,4)  or  the  equivalent  equations  (3,11).  Its  solution  is,  however, 
not  the  least  squares  solution  but  an  approximate  solution  of  unknown 
quality. 


Initial  approximations  to  the  solution  also  aan  be  obtained  by 
other  falsifications  in  addition  to  the  one  described,  or  instead 
of  it.  Such  falsifioations  are,  e.g.,  assumptions  that  oertain 
observations  are  error  free,  that  some  correlations  are  gero,  that 
some  model  parameters  have  prescribed  values,  eta. 

4 ,  EXAMPLES .  The  first  example  is  a  case  involving  transformation 
between  polar  and  cartesian  coordinates.  We  shall  compare  results 
that  are  obtained  using  the  approach  of  the  previous  section  with 
results  that  are  obtained  by  following  suggestions  by  other  authors, 
in  data  processing  literature  one  finds  different  suggestions.  The 
simplest  one  is  to  treat  the  problem  after  transformation  as  if  the 
transformed  quantities  were  observed,  it.  is  clear  from  the  discussions 
in  Section  2  that  such  an  approach  does  not  produce  the  least  squares 
solution,  i.e. ,  it  does  not  minimise  W{||c||},  even  if  the  transfor¬ 
mation  is  linear.  The  most  sophisticated  suggestion1*8'10  is  to 
apply  the  transformation  (3.18)  to  R,  i.e.,  to  solve  the  system  (3.17). 
As  we  have  seen  in  the  previous  section,  this  approaoh  yields  the 
least  squares  solution  only  if  the  transformation  Y«v(X)  is  linear. 

The  following  example  illustrates  the  practical  consequences  of  such 
a  problem  falsification. 


Let  the  observations  be  distances  rj.  and  asimuth  angles  4>j.,  and 
let  the  model  equations  represent  a  straight  line  in  cartesian 
coordinates.  Then  the  model  equations  are  in  terms  cf  the  original 
observations 

f  r^inp^  -  a  -  br^oos^  -  0 


r2sind2  -  a  -  br^cos^  -  0 


F(r,(|>>a,b)  -  < 


(4.1) 


Lr  sin$  -  a  -  brcos*  ■  0 
n  n  n  rn 

The  transformation  of  the  obnervations  into  cartesian  coordinates  are 
xi  -  rJL  cos^i  , 

yi  “  “in^  »  i  ■  1*  2,  n  ,  (4.2) 
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and  the  nodal  aquations  ara  In  tarns  of  tha  transfomad  obsarvatlons 


H(X,Yj«,b)  -  < 


Yi  -  a  -  bxx  -  0 


y2  -  a  -  bx2  -  0 


,n  -  «  -  b»n  -  0 


(4.3) 


the  Jacobian  matrix  of  the  transformation  is 


where 


<4.4) 


3(Vyi} 

Ji  "  f(r“^T 


cos<)^  -r^sin^j 
sin^  r^cos^ 


(4.5) 


We  assume  for  simplicity  that  all  observations  are  independent 
with  estimated  standard  errors  erj,  and  e^ ,  respectively.  Then  the 
estimated  variance- covariance  matrix  R  is  the  diagonal  matrix 
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The  transformed  variance- covariance  matrix  Q  is  according  to  equation 
(3.18)  the  block  diagonal  matrix 


g  -  jR-r  - 


where 


(4.7) 


V' 


2  2  2 


(*ri"eJiri)8inV°**i 
2  2 


<eri"Vri)8in*i°°L*i  *ri*in  *i+%iriOOB  *i 


(4.8) 


For  a  numerical  example  we  take  the  ten  points  listed  in  Table 
X  as  observations  and  assume  that  their  standard  errors  are 


eri  -  0.C48,  e^  -  27.5%  i  -  1,  2,  . . . ,  n.  (4.9) 

We  made  three  adjustments.  First,  the  t, 4-data  were  used 
together  with  the  model  equations  (4.1).  Xn  the  uaoond  adjustment, 
the  x,y-data  were  used  together  with  the  model  equations  (4.3)  and 
the  transformation  function  (4.2)  in  a  utility  program1®  based 
on  the  normal  equations  (3.11).  The  results  of  both  adjustments  were 
identical,  as  they  should  be,  and  they  are  listed  in  Table  XX.  The 
listed  standard  errors  of  the  parameters  are  the  square  roots  of  the 
diagonal  elements  of  V.,  computed  with  formula  (3.8).  The  correlation 
coefficient  c^  is  the  off-diagonal  element  of  the  correlation  matrix 
Ct>  defined  by 

ct  ■  d;*  vt  ■£*  14-101 


where  Ut  -s  the  diagonal  matrix  of  Vf  The  standard  error  of  weight 
one  is  defined  by 


-  t 


n-p 


T  ■ 
C  R 


^Cl* 


1 

n-p 


W] 


(4.11) 


142 


Figure  la  shows  the  result  of  the  adjustment  in  the  $ , r-plane , 
i.e..  in  the  plane  of  the  original  observations -  The  accuracies  of 
the  observations  are  indicated  by  error  ellipses  around  the  observed 
points.  The  adjustment  is  indicated  by  connecting  the  observed  points 
with  the  corresponding  corrected  locations  on  the  fitted  ourve.  The 
figure  shows  that  all  adjustments  are  in  the  direction  of  largest 
uncertainties. 

Figure  lb  shows  the  same  result  in  the  x,y-plane.  The  accuracies 
of  the  transformed  observations  are  again  indicated  by  error  ellipses, 
corresponding  to  the  transformed  variance-covariance  matrices  Qi*  In 
this  presentation  the  adjustments  seem  to  be  in  directions  other  than 
those  with  largest  uncertainties.  This  is  typical  for  nonlinear 
transformations  of  observations.  The  object  of  the  fitting  is  to 
minimize  residuals  of  the  original  observations.  The  presentation  in 
the  x,y-plane  is  distorted  by  the  nonlinearity  of  the  transformation. 

In  a  third  adjustment  wo  used  the  x, y-data,  the  model  equation 
(4.3),  and  the  variance-covariance. matrix  Q,  defined  by  equation  (4.7). 
The  treatment,  suggested  by  Deming  and  other  authors,  was  described 
in  Section  3,  equations  (3.16)  through  (3.18),  as  a  falsification  of 
the  problem.  The  numerical  results  of  this  adjustment  are  listed  in 
Table  II.  They  are  different  from  the  previous  results,  and  the 
increase  of  m  indicates  that  the  -olution  is  not  optimal.  We  notice 
also  that  the°correlation  coefficient  c  has  changed  its  magnitude 
and  sign. 

Figure  2b  shows  the  results  of  the  adjustment  in  the  x,y-plane. 

It  indicates  that  the _ adjustment  would  indeed  be  optimal,  if  x,y  were 
the  observations  and  Q  was  their  variance-covariance  matrix.  However, 
when  the  same  results  are  plotted  in  the  4,z-plane,  Figure  2a,  then 
it  becomes  obvious  that  the  adjustment  has  not  aahiaved  the  goal  to 
minimize  the  residuals  of  the  original  observations  <f>,r.  The  treatment 
of  transformations  of  observations  in  this  form  is  a  falsification 
of  the  problem.  The  results  are  approximations  to  the  least  squares 
solution,  but  since  the  quality  of  the  approximations  are  not  known, 
they  may  be  useful  only  as  initial  approximations  for  a  least  squares 
algorithm.  However,  in  a  case  like  this  example,  an  initial  approxi¬ 
mation  could  be  simpler  obtained,  e.g.,  graphically  by  drawing  a 
straight  line  in  the  x,y-plane  through  the  observations. 

Next,  we  present  an  example  for  the  linearization  of  parameters. 
Let  the  model  equation  be 

y  -  AxB  exp  (^)  »  0,  (4,12) 


N»3 


1 *v."  M,  Mfrtl 


where  x  end  y  are  observations  and  A,  B,  and  C  are  nodal  parameter a. 
An  equivalent  model  formulation  la 


Iny  -  a  -  b  Inx  -  —  »  0. 


(4.13) 


In  equation  (4.13)  the  parameters  a,  b,  and  c  enter  linearly.  One 
can  expect  c.  much  better  performance  of  aolution  algorithms  if 
aquation  (4.13)  ia  uaed.  The  parameter  transformation  ia  in  this 
example 


A  -  e*. 


B  «  b, 


(4.14) 


C  »  a, 


and  the  Jacobian  matrix,  needed  in  aquation  (2.23)  la 


j  m  0  0 

"  1  0  10 

9(a,b,c)  l 

\  o  0  1 


(4.15) 


Another  example  ia  the  trigonometric  model 


y  -  A  ooe  ■  0. 


(4.16) 


An  equivalent  model  ia 


y  -  a  ain(cx)  -  b  coa(cx). 


(4.17) 


The  corresponding  parameter  transformation  is 

a  -  A  oin(B/C), 
b  -  A  cob(B/C) , 
c  ■  1/C, 


(4.18) 
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with  the  Jacobian  matrix 


i 

\ 


*Shi*x£L  .  I  -1  _ 

3(a,b,c)  1  3 (A,B,C)  1 


j  sin (B/C)  (A/C) oob (B/C) 

J  Cot  (B/C)  -(A/C) tin (B/C) 

\  0  0 


- (AB/C2) cot (B/C)  V“l 
(AB/C2) tin (B/C)  ) 

-i/c2  / 


(4.19) 


In  this  example,  the  model  (4.17)  it  linear  only  with  retpect  to  two 
parameter!.  However ,  the  difference  o Z  numerical  treatment!  of  the 
problem  it  dramatic  if  one  uset  equation  (4.16)  or  equation  (4.17), 
reapectively.  In  numerical  experiments  we  found  that  in  order  to 
achieve  convergence,  one  had  to  start  with  parameter  values  A,B,C 
within  few  peraent  of  their  least  squares  values.  Using  the  parameters 
a,b,o  and  the  model  equation  (4.17),  one  achieves  fast  convergence, 
e.g.,  with  the  initial  values  a-b-O. 


5,  SUMMARY  AND  CONCLUSIONS.  Manipulations  of  model  equations 
that  produce  simpler  but  equivalent  equations  can  greatly  facilitate 
the  preparation  of  the  problems  (e.g. ,  computer  programming)  for 
utility  routines.  The  manipulations  can  also  improve  the  performance 
of  numerical  algorithms.  If  the  manipulations  are  merely  algebraic 
and/or  involve  nonlinear  transformations  of  the  model  parameters, 
then  their  application  is  straight  forward  and  thair  implementation 
simple.  If,  however,  the  manipulations  include  transformations  of 
observations,  then  one  has  to  transform  also  the  normal  aquations 
correspondingly.  Neglect  of  this  transformation  falsifies  the  problem 
and  produces  results  that  are  of  unknown  quality  and  equally  reliable 
as,  e.g.,  a  graphical  construction  of  a  fitting  curve.  A  correct 
implementation  of  transformations  of  observations  requires  the  pro¬ 
gramming  of  the  transformation  function,  including  its  first  and  second 
order  derivatives.  It  also  does  not  improve  the  performances  of 
algorithms.  Therefore,  in  most  cases,  it  is  more  efficient  to  formulate 
the  model  equations  in  terms  of  the  original  observations,  thereby 
avoiding  the  programming  of  the  transformation  function. 


The  need  for  seaond  order  derivatives  of  the  model  equations  has 
been  often  overlooked.  In  order  to  avoid  the  programming  of  these 
derivatives,  most  authors  suggest  to  use  a  first  order  Gauss-Newton 
algorithm  for  the  solution  of  the  normal  equations,  instead  of  a 
second  order  Newton-Raphaon  algorithm.  The  performance  of  the  former 
may  be  often  comparable  to  the  latter,  because  even  with  more  iterations, 
the  computing  effort  can  be  less  due  to  the  simpler  aquations.  Second 
order  derivatives  of  the  model  equations  (and  of  the  transformation 
function)  are,  however,  needed  to  compute  the  linear  terms  in  formulas 
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for  variance  estimates  of  the  results.  Thalr  neglect  cannot  be 
justified  curaory  by  the  argument  that  linearised  model  equation*  are 
already  second  order  accurate  and,  therefore,  their  second  order 
derivatives  are  not  needed,  it  can  be  shown  that  the  linearised  normal 
equations  do  contain  these  derivatives  and,  therefore,  are  needed  in 
the  linearised  variance  propagation  formula.  Formulas  for  variance 
estimates  that  do  not  contain  second  order  derivatives  are  less  than 
first  order  accurate. 


TABLE  I.  OBSERVATIONS  $  AND  r  AND  CORRESPONDING 
CARTESIAN  COORDINATES 


206. 6* 

0.559 

-0.50 

-.025 

26.6° 

1.342 

1.20 

0.60 

26.6* 

!  2.236 

2.00 

1.00 

26.6* 

3.354 

3.00 

1.50 

26.6* 

4.472 

4.00 

2.00 

123.7* 

1. 803 

-1.00 

1.50 

92.9* 

1.952 

-0.10 

1.95 

68.2* 

2.693 

1.00 

2.50 

52.4* 

4. 100 

2.50 

3.25 

42.0* 

6,727 

5.00 

4.50 

TABLE  II.  ADJUSTMENT  RESULTS 

Cast  1  »nd  2  (Original  and  Transformed  Problem) 
a  -  0.381  ±  0.298 

b  -  1.141  ±  0.744  -  0.015065 

m  -  1.24541 
o 

Cage  3  (Falsified  Problem) 
a  *  0.680  ±  0.407 

b  -  1.837  ±  0.259  -  -0.568659 

m  «  1.75646 
o 

The  standard  error  of  weight  one,  mQ,  is  not  included  in  the  standard 
errors  of  the  parameters.  J 
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Figure  2a.  Falsified  Adjustment  in  x,y  Space. 

The  data  are  shown  with  their  one  standard  error  ellipses 
and  the  adjusted  curve  is  shown  with  one  standard  error 
confidence  limits.  The  same  results  are  shown  in  Figure 
2b  in  the  cartesian  x,y-plane. 
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Figure  2b.  Falsified  Adjustment  in  x,y-space. 

Tha  transformed  data  are  ahown  with  thair  one  standard  error 
ellipses  and  tha  adjusted  lina  ia  ahown  with  ona  standard 
arror  confidence  limits.  Tha  aama  raaulta  are  ahown  in  Figure 
2a  in  the  4>,r-plane  of  observations. 
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APPENDIX 


We  provide  a  set  of  iteration  formulas  that  are  derived  from 
the  Newton  equation  (3.6)  by  algebraic  manipulations.  First,  we  define 
the  following  matrices t 


0  -  (rX>_1 

r  -  [I+ARU^F)^]"1 
s  r*  (ac-rf3ofw] 

O  XX 

E1  "  r*  lM,j8Ft+AR(KTF)Xt] 

■  «*«tX  - 

D1  *  <*>«  - 


The  iteration  equations  are 
nt  -  fJ;g(fxc-f)  +  dqe0 

K+K  -  3(FxC-F)+0(Ft+Fj{R(KTF)xt]T-aFxR(K,rF)xxe 


(A.l) 
(A. 2) 
(A. 3) 
(A.4) 
(A.  5) 
(A.6) 
(A.7) 
(A. 8) 

(A.9) 
(A. 10) 

(A. 11) 


Numerical  experiments  have  shovm  that  the  oonvergenoe  of  the 
iteration  in  enhanced  if  the  equations  are  used  in  e  subiteration 
mode  by  Iterating  alternatively  on  the  parameters  and  residuals, 
respectively.  For  parameter  subiteration  only  equations  (A.9)  and 
(A. 10)  are  used,  assuming  c=0.  For  residual  subiteration  one  sets 
T  =0  and  uses  equations  (A. 10)  and  (A. 11). 
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Xn  tha  varinnoe  formula  (3.8)  ona  usee  M,  defined  by  aquation 
(A. 8)  and 


s  -  r*ary  +  DQrA.  (a.  12) 


Anothar  equivalent  aat  of  Newton- Raphaon  i tar at ion  aquationa  ara 
given  in  reference  13.  Mona  of  the  aata  ara  numerically  superior  to 
tha  other,  and  both  raquira  aubitarationa  of  parameters  and  residuals 
for  affioianoy. 

Gauss-Newton  itaration  aquationa  can  ba  obtained  from  Mawton- 
Raphaon  iteration  aquationa  by  aatting  all  aaoond  order  darivativaa 
zero.  Tha  convergence  of  Qauaa-Nawton  algor ithma  la  inferior ,  but 
in  aoma  applications  they  have  a  larger  domain  of  convergence. 

Iteration  aquationa  for  leant  aquarea  problamn  with  tranaformationa 
of  obeervationa  can  be  obtained  from  the  formulae  in  this  Appendix 
by  aubatituting 


Q  for  R 
A  for  C 

and 

B  for  (k Tr)xx 


Expressions  for  Q,  A,  and  2  in  terms  of  the  model  and  tha  transformation 
functions  ara  given  in  Section  3,  aquationa  (3.13),  0.14),  and  (3.15). 
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ANALYSIS  OF  DATA  VITH  THE  NONLINEAR  LEAST  CHI 
SQUARE  ALGORITHM 

Richard  L.  Moore 

US  Amy  Armament  Research  and  Development  Command 
System  Evaluation  Office 
Dover,  NJ  07801 

ABSTRACT,  This  paper  reports  on  the  use  of  the  least  chi-square 
algorithm  for  fitting  data  to  non-linear  functions  of  the  parameters, 

A  well  known  computer  program  of  the  National  Institutes  Health, 
SAAM-27,  has  been  modified  to  use  this  algorithm.  Comparison  of  the 
ordinary  least-squares  algorithm  with  the  new  algorithm  have  been 
made  on  four  different  problems  as  follows:  Pressure  waves  In  gun 
chambers,  control  of  aircraft  yaw,  a  biomedical  kinetic  reaction 
involving  four  measured  components,  and  a  very  non-linear  nuclear 
reactor  kinetics  problem.  The  preliminary  results  indicate  that  the 
least  chi-square  algorithm  is  practicable,  that  the  computing  time  is 
increased  for  short  problems,  but  evens  out  for  long  problems. 

The  least  chi-square  algorithm  appears  to  be  less  failure  prone 
than  least  squares  and  a  test  has  been  inserted  in  the  program  to 
preclude  any  iterations  which  might  tend  toward  maximizing  the 
autocorrelations  as  could  occur  when  their  initial  value  is  large. 

I.  INTRODUCTION.  It  is  accepted  procedure  in  analysing  the 
goodness-of-fit  of  experimental  data  to  a  theory  Which  is  nonlinear 
in  the  adjustable  parameters  to  estimate  whether  the  residuals  are 
consistent  with  being  drawn  from  a  normally  distributed  population  in 
a  random  sequence.  A  common  statistic  to  test  the  random  sequence 
hypothesis  is  the  sum  of  the  squares  of  the  normalized  autocorrelation 
coefficients  frequently  called  the  Box-Pearce  test.  However,  if 
these  tests  indicate  a  lack  of  agreement  with  the  hypothesis,  no 
rationale  has  been  available  to  modify  the  parameters  to  obtain  a 
better  fit.  A  solution  to  this  problem  has  been  provided  by  the  use 
of  a  least  chi  square  algorithm  which  estimates  the  parameters  which 
give  the  greatest  probability  that  the  residuals  arise  from  a  popula¬ 
tion  with  variance  ,  and  are  sampled  from  a  random  sequence. 

II.  SUMMARY  OF  MATHEMATICS. 


Following  the  notation  of  Altken  (1)  and  as  previously  derived  by 
Moore  (2,  3),  we  define  the  following: 

The  transpose  of  a  vector  or  matrix  is  Indicated  by  a  '  on  the 
symbol  u' . 


u  is  the  vector  of  observed  values. 

y*  Is  the  vector  of  theoretical  values  corresponding  to  u. 

6*  Is  the  vector  of  the  estimates  of  the  unknown  parameters. 

P*  is  tha  matrix  of  the  partials  of  y*  with  respect  to  $*. 


The  calculaton  of {66}  is  done  from  the  following  expression  which 
1c  the  same  as  the  usual  non-linear  least  square  lnteratlon  except 
for  the  matrix  T  . 

[58*1  «  [p*'  r  p*!-1  p*'  r  u*. 


In  the  usual  case,  T  equals  1. 

This  expression  has  been  programed  into  the  Simulation  And 
Analysis  Modeling  (SAAM-27)  (4,  5)  program  of  Berman  et  al,  by  post- 
multipying  P*1,  by  T  ,  and  letting  the  program  proceed  from  that 
point,  with  the  data  for  pa*  being  replaced  in  the  memory  by  the 
product  P*T.  The  usual  iteration  continues  from  this  point.  The 
computer  program  resulting  from  this  change  has  been  designated  for 
control  purposes  as  SAACH,  and  has  been  tested  on  the  CDC  6600  at 
ARRADCOM,  Dover,  to  determine  the  following  questions: 

1.  How  much  change  is  there  In  the  final  parameter  estimates? 

2.  What  change,  if  any,  ia  there  in  the  number  of  iterations? 

3.  What  change  is  there  in  the  time  per  iteration? 

111.  EXAMPLES .  Four  problems  of  different  origin  which  use 
different  mathematical  models  have  'been  run  on  the  SAACH  program  to 
answer  the  above  questions.  In  the  first  example:  Gun  Chamber  Pre¬ 
ssure  Waves,  the  mathematical  model  used  is  the  ouperpositlon  of  two 
pressure  waves  generated  by  analytic  models  in  the  program,  with  the 
adjustment  of  up  to  eight  parameters  to  obtain  the  best  fit  to  ob¬ 
served  data.  In  the  second  example,  an  aircraft  control  system 
simulation,  the  mathematical  model  is  a  set  of  four  linear  differen¬ 
tial  equations,  simulating  the  Yaw  Damper  system  on  an  aircraft. 

These  equations  were  solved  by  a  special  procedure  developed  for 
SAAM-27  by  Berman  et  al.  (6),  with  up  to  four  adjustable  parameters. 
In  the  third  example,  a  biomedical  problem  furnished  as  a  test  case 
by  Miss  Rita  Straub  of  Brookhaven  National  Laboratory,  the  mathema¬ 
tical  model  was  a  set  of.  seven  coupled  linear  differential  equations 
with  five  adjustable  parameters;  this  was  solved  by  the  same  method 
as  used  in  the  second  case.  In  the  fourth  and  final  example:  KEWD 
Kinetics,  s  simulation  of  the  nuclear  reactor  transients  of  the 
Kinetic  Experiment  Water  Boiler,  the  mathematical  model  was  an  ex¬ 
tremely  non-linear  Bet  of  coupled  differential  equations  as  described 
by  Hetrick  and  Gamble  (7).  These  equation  were  integrated  by  the 
fourth  order  Runge-Kutte  integration  procedure  of  SAAM-27,  with  only 
one  ad Jut  table  parameter. 
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III-l  Cun  Chamber  Pressure  Waves.  Unusual  pressure  waves  sug¬ 
gestive  of  an  acoustic  wave  superposed  on  the  normal  gun  chamber 
pressure-time  curve,  have  occurred  in  tests  of  the  XM211  propellant 
charges  at  zone  3  for  the  M101  projectile  in  the  155mm  gun, 
(Knutelskl,  (8)).  Analysis  of  these  waves  was  Initiated  by  Mr.  B. 

G.  Knutelskl  of  the  Large  Caliber  Weapon  Systems  Laboratory  using  the 
SAAM-27  program.  A  parallel  analysis  was  made  by  the  author  using 
the  SAACH  program.  In  order  to  have  as  little  bias  as  possible 
Injected  into  the  comparison,  the  Knutelskl  model,  data,  and 
procedure  was  followed  as  closely  as  possible.  The  resulting  data 
fit  was  later  improved  by  using  more  data  and  Improved  models.  The 
history  of  this  analysis  is  Important  because  It  illustrates  the 
problems  which  arise  when  no  prior  knowledge  is  available  about  the 
best-fitting  model.'  (This  example  Is  the  only  one  of  the  four 
examples  for  which  prior  knowledge  was  not  available.) 

The  first  case  wab  run  using  the  data  shown  as  asterisks  in  Figure 
1.  This  figure  shows  the  theoretical  fit  by  the  following  model: 


P  -  P(l)sin{2ir(P(2)t  +  P(3))} 

+  P(4)sin{2n(P(5)t  +  P(6))}  . 


Fig.  2  shows  the  theoretical  fit  by  the  same  model  ae  above  using  tha 
Least  Chi  Program  (SAACH)  with  five  autocorrelation  coefficients  (BGK 
1.101).  Table  1  indicates  the  number  of  iterations  to  convergence 
and  the  final  values  of  the  parameters  (the  initial  values  were  the 
same).  The  value  of  the  sum  of  the  squares  (Xj2)  i8  given  for 
comparison,  as  well  as  the  autocorrelation  coefficients  up  to  rank  5. 
Case  BCK  had  slightly  lower  values  of  Bums  of  squares,  but  the  chi 
square  was  much  smaller  for  RGK  1.101.  (The  symbol  X  will  be  used 
for  the  greek  letter  Chi  for  the  rest  of  this  report.) 

Because  not  all  the  data  points  available  were  used  In  this 
preliminary  analysis,  additional  data  were  obtained  and  entered  into 
the  computer  using  the  same  model  and  same  initial  conditions  as  in 
the  previous  runs.  In  this  case  (BGK  3.002)  the  least  squares 
iteration  stopped  at  seven  iterations;  as  shown  in  Figure  3,  the  fit 
was  poor  and  the  convergence  obviously  false.  The  least  chi  square 
iteration,  BCK  3.102,  using  the  same  data  terminated  at  14  iterations 
with  an  obviously  better  fit  (Fig  4),  but  yet  not  a  good  eyeball  fit. 
The  results  of  both  eases  are  also  shown  in  Table  1.  The 
autocorrelation  coefficients  are  largo  for  case  3.102,  and  indicate 
the  general  lack  of  fit. 


Because  of  chi*  obvious  lack  of  fit  In  BCK  3.102,  tho  modal  was 
changed  to  the  following. 


P  “  P^  exp  (  -  ,/2o1a) 

+P2  exp  {  -  (t-t2)  V2a»*  }  X  ein{  2tt£  <t-t3)  +  ir/2} 


Throe  parallel  cases  were  computed  once  the  fit  was  good  enough 
to  parmlt  Iteration.  Because  of  computing  difficulties  which  arose 
when  trying  to  converge  on  alx  or  seven  parameters,  the  Iteration  wee 
Initially  restricted  to  four  parameters i  Once  the  fit  was  good  and 
had  converged  using  these  four  parameters,  their  final  values  were 
used  as  initial  values  for  a  six-parameter  fit.  Finally,  all  eight 
parameters  were  allowed  to  vary. 


The  results  of  this  series  of  analysis  are  plotted  in  Figs  5,  6, 
and  7.  Tho  case  numbers  are  BGK-3. 30356301-0,  3 . 303565 11—5  end 
3.30356511-10  respectively.  The  first  has  no  autocorrelation 
coefficients;  the  second,  5;  and  the  third,  10.  The  parameters  for 
these  cases  are  given  in  Tahir  2,  (note  that  the  last  three  digits 
only  of  the  identifier  are  used  here).  The  estimated  errors  are  the 
estimated  standard  deivatlons  based  on  the  value  of  the  sum  of  the 
squares.  In  the  case  of  511-5  and  -10,  the  value  of  was  used 
rather  than  the  sum  of  squares.  The  statistical  validity  of  this 
procedure  has  not  yet  been  established. 


Results  shown  in  Figures  5,  6,  end  7  indicate  that  the  apparent 
fit  to  the  data  is  best  for  the  case  of  five  autocorrelations,  (Fig 
6).  In  this  figure  the  autocorrelations  were  weighted  higher  than  in 
Fig  7,  where  ten  autocorrelations  were  used,  and  of  course  much  higher 
than  in  Figure  5,  where  no  weight  was  given  to  the  autocorrelations. 

It  is  clear  from  Table  2  that  ordinary  least  squares,  case  301 
indicates  a  small  fractional  standard  deviation  as  compared  to  the 
other  two  cases,  but  yet  the  fit  to  the  data  is  not  as  good  as  seen 
from  its  plot,  (Figure  5). 

The  last  row  of  Table  2,  gives  the  velucs  of  ,  the  experi¬ 
mental  variances  assumed  for  these  casos.  These  were  arbitrary 
numbers  in  this  case,  because  the  precision  of  the  measurement  system 
is  probably  much  greater  than  the  value  given  i.e.,  the  variances 
should  be  smaller.  However,  if  smaller  values  were  used,  such  as 
when  caoe  511-10  le  compared  to  511-5,  the  weight  on  the  sum  of  the 
squares  ia  greater  but  the  goodness  of  fit  appears  to  decrease* 
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Table  2  also  shows  the  effect  of  least  chi-square  In  terms  of 
number  of  Iterations,  and  computing  time.  When  five  autocorrelations 
were  used,  as  in  case  51 1-5,  only  a  small  increase  in  number  of 
iterations  is  found  and  a  moderate  Increase  in  computing  time  as  com¬ 
pared  to  301,  If  ten  autocorrelations  were  used,  as  in  511-10,  the 
number  of  iterations  increased,  and  the  time  increase  was  1.8  times 
greater,  giving  about  double  the  increase  in  time  for  double  the 
number  of  autocorrelation  coefficients. 

Table  3  shows  the  autocorrelations  up  to  order  20  for  the  three 
cases.  The  values  of  X^2,  X2  ,  and  Xx2  for  the  number  of 
autocorrelations  used  (0,  5,  10)  is  shown  in  the  last  rows  of  this 
table. 

The  difference  in  the  assumed  experimental  variances  accounts  for 
most  of  the  difference  between  Xj2  for  cases  511-5  and  511-10.  If 
the  experimental  variances  had  bssn  the  same,  X}2  would  have  been 
62.96  or  57.62.  Case  511-5  appears  (in  the  figures)  to  fit  better 
because  the  first  five  autocorrelations  as  well  as  most  of  the  later 
autocorrelations  are  smaller. 


I1I-2  Aircraft  Control  Systems.  The  block  diagram  of  a  typical 
problem  of  this  type  is  shown  in  Fig  8.  To  optimize  the  design  four 
parameters  ma^  be  adjusted  to  give  the  best  fit  to  a  desired  response 


These  parameters  are 


and  Kg  .  These  correspond 
to  the  parameters  L(0,4),  L(4,l),  br(4,2)  and  1(4.3).  A  previous 
analysis  of  this  example  Cising  SAAM-23  was  available.  A9  a  result,  a 
completely  unbiased  comparison  of  least  squares  and  least  chi  squares 
prodecures  was  difficult  to  ensure.  Two  different  approaches  were 
used  on  this  example.  First,  the  "data" — corresponding  to  the  desired 
curve — was  used  "as  is"  for  comparison  with  the  calculated  response. 
Second,  a  vector  of  a  random  sequence  of  normally  distributed  errors 
from  a  population  with  variance  of  (.033)2  was  added  to  the  data 
vector  to  simulate  the  effects  of  sampling  error;  this  may  be 
considered  to  represent  an  allowable  error  or  tolerance  in  fitting 
the  curve. 


In  the  first  approach,  the  cases  to  be  compared  are  2-6  and  4. 
Case  1-6  was  a  reference  run  which  adjusted  four  parameters,  and 
started  near  to  the  final  values.  It  Iterated  three  times  and  took 
23.6  sec  to  complete.  A  similar  case,  2-6  used  the  same  starting 
point  and  used  six  autocorrelations  coefficients.  It  failed  to  im¬ 
prove  the  fit  in  but  one  iteration,  primarily  because  it  attempted  to 
increase  the  autocorrelations  in  its  attempts  to  improve  the  fit. 
(Several  cases  of  this  type  were  found  which  led  to  a  modification  in 
the  least  chi-square  algorithm,  to  be  discussed  later).  The  data  on 
the  parameters,  autocorrelations,  and  chi  squars  are  given  in  the 
first  column  of  Table  4.  The  fit  to  these  data  are  shown  in  Fig  9. 


In  an  attempt  to  understand  this  problem  various  strategist,  were 
tried,  but  the  results  were  nearly  ell  similar.  Case  4  and  4.1  are 
typical.  In  these  comparisons  the  number  of  adjustable  parameters 
was  reduced  to  three,  the  value  of  the  experimental  variance  was 
increased  to  1.0  and  the  rank  of  the  autocorrelations  was  taken  as  5, 
6,  and  12.  In  the  least  squares  case,  case  4,  as  indicated  in  Table 
8,  the  calculation  converged  in  six  iterations,  taking  11.2  Bee.  In 
case  4.1,  with  the  same  Initial  point,  the  three  attempts  iterated 
for  six  iterations,  for  the  same  time,  11.2  sec,  but  because  the 
autocorrelations  were  large,  and  the  value  of  X2*  was  large 
compared  to  Xj*,  they  *n  eventually  diverged  from  a  good  tit. 

These  results  Indicated  that  the  algorithm  was  not  reliable  when  the 
autocorrelations  were  large  at  the  outset.  To  correct  this,  an 
Internal  algorithm  will  be  added  ao  that  value  of  «  which  is 
1/(X^  -2X2^)  will  not  be  allowed  to  be  greater  than  .5.  It  is 
believed  that  this  changa  will  prevent  situations  of  this  kind  from 
arising  in  the  future,  but  the  effect  of  this  change  has  not  yet  bean 
fully  tested.  To  determine  whether  the  least  chi-square  technique  is 
valid  for  the  Yaw  Damper  calculation,  the  second  approach,  the 
addition  of  Monte  Carlo  errors  to  the  desired  response  curve,  was 
used  as  a  teat. 


For  this  second  approach,  a  random  sequence  was  added  to  each  of 
the  data  points.  The  value  of  o|  was  set  at  (.033)^,  six  autocorra 
latlons  were  used  for  the  problem  which  wbb  Identified  as  CONRLM 
4.011-6.  Another  run  was  used  on  the  same  data  with  the  standard 
least  squares  algorithm.  Fig.  10,  (CONRLM  4.012)  showa  the  fit 
obtained  for  the  data  and  is  typical  of  the  results.  Table  4  showa 
the  number  of  iterations  for  each  cage.  It  took  4  iterations  for  the 
ordinary  algorithm  to  converge,  and  only  two  for  the  least  chi-sq. 
algorithm  with  six  autocorrelation  coefficients  (CONRLM  4.011-6). 

The  time  for  one  iteration  waa  8.2  and  8.5  aec  respectively.  (Part  of 
the  Increase  in  time  for  the  least  chi-square  case  waa  due  to  several 
attempts  in  both  iterations  to  improve  the  fit  by  reducing  the  step 
sire.)  As  shown  in  Table  4  the  parameters  L(0,4),  L(4,2)  and  L(4,3) 
appear  to  be  different  by  significant  amounts,  and  the  difference 
in  the  "significance"  of  the  two  results  is  considerable.  (The 
autocorrelations  for  case  4.011-6  appear  well  within  the  random 
range. ) 


Ill' *3  Brookhaven  Example.  A  sample  test  case  wa*  received  from 
Miss  Rita  Straub  of  Brookhaven  National  Laboratory.  The  exact  nature 
of  the  problem  was  unspecified  but  from  the  form  of  the  differencial 
equations  given  in  table  5,  it  appeals  to  be  a  kinetic  problem  in 
which  the  materiel  in  component  one  decays  into  components  two  to 
five,  and  component  two  may  change  into  component  one.  Component 
seven  la  composed  of  components  three,  four,  and  five.  Although  the 
”S"  and  "K”  parameters  may  actually  be  unknown,  they  ware  assumed 
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known,  because  the  present  version  of  the  program  will  not  Iterate 

either  type  of  linear  parameters  with  the  least  chi-square  algorithm. 

The  data  were  available  for  the  amount  of  components  1,  3,  4,  and  6 
as  a  function  of  time,  (where  component  6  is  the  sum  of  components  1, 
2,  and  5). 

Both  the  run  with  no  autocorrelations  and  the  run  with  5 
correlations  (KJE  1.0023-5),  took  7  iterations  to  converge.  The 
results  for  the  two  cases  are  compared  in  Table  6.  Since  the  valuo 
of  (31.24)  is  large  compared  to  X2*  (3.04),  the  major 

emphasis  in  this  case  was  on  reducing  the  sum  of  squares,  and  thus  It 
is  similar  to  the  case  run  with  no  weight  on  the  autocorrelations. 

As  would  be  expected,  there  is  only  a  small  difference  between  the 

final  values  of  the  parameters  of  the  two  cases.  Figs  10,  11,  and  12 

show  the  graphs  of  the  data  fit  to  the  components  1,  3,  and  4. 
(Component  6  shows  an  exact  fit  to  data  points  and  therefore  a  graph 
of  this  component  is  not  provided.) 

IXI-4.  Reactor  Kinetics  Example.  This  example  illustrates  two 
things:  First  the  use  of  the  least  chi-square  algorithm,  and  second 
a  good  fit  between  data  and  a  physically  incorrect  modal.  Hetrick 
and  Gamble  (7)  proposed  a  non-linear  feed-back  term  proportional  to 
the  energy  in  the  reactivity  of  the  KEWB  reactor  to  describe  the 
effect  of  void  on  reactor  shutdown.  Although  this  model  gives  a  good 
fit,  later  experiments  (9)  where  the  void  amount  was  inferred  from 
measurements  and  where  the  thermal  effects  on  reactivity  were  also 
carefully  measured,  showed  that  shutdown  was  due  to  thermal,  not  void 
effects.  In  the  simulation,  the  effect  of  the  energy  on  void  forma¬ 
tion  was  simulated  by  the  parameter  L(ll,  1).  The  functions  corres¬ 
pond,  in  numerical  order,  to  the  functions  used  in  the  simulation: 

(1)  Nuclear  reactor  power  level,  (2)  Mean  temperature,  (3)  Mean  void 
volume,  (4)-(9)  Delayed  neutron  groups,  (10)  Not  used,  (1.1)  Energy 
released  to  that  time.  The  result  of  the  iterations  is  shown  in 
Figure  13,  a  logarithm  plot  of  theoretical  and  experimental  nuclear 
power.  In  Table  7,  three  different  cases  are  shown: 

Case  1.003-0  was  ordinary  least-squares.  The  values  of  the 
autocorrelations  and  chi-squares  are  shown  for  comparison  with  the 
other  two  cases.  Case  1.005-3  used  three  autocorrelations  with  a 
small  value  of  the  experimental  variance  thus  resulting  in  a  large 
value  of  Xj^.  Both  case  1,007-6  and  1.003-0  use  1  x  10^  for  the 
experimental  variance  thereby  reducing  the  emphasis  on  the  sum  of  the 
squares  of  the  errors.  All  of  these  runs  took  four  iterations  to 
converge. 

Cases  1.003-0  and  1.005-3  give  almost  exactly  the  same  results. 

On  comparing  1.003-0  with  1.007-6,  a  difference  is  found  in  the  value 
of  the  adjustable  parameter  L(ll,l).  The  value  of  chisquare  total  la 
smaller  for  1.007-6,  and  thus  this  result  would  be  chosen  over  that 
of  the  other  case. 
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The  value  of  the  chi-square  for  the  autocorrelation  (X2^)  is 
much  smeller  for  caea  1.007-6,  although  Xj*  is  slightly  larger  for 
the  same  case — thus  Illustrating  the  trade-off  between  getting  the 
minimum  as  In  ordinary  least  squares,  and  reducing  the  autocorrela¬ 
tions  as  in  least  chl-squarea.  The  data  for  Case  1,003  show  the 
values  for  K(l)  to  R(6)  for  comparison  purposes.  The  data  show  that 
the  sum  of  squares  does  not  increase  from  one  to  other  appreciably, 
but  Xg^,  Che  Box-Pearce  statistic,  does  change  appreciably.  Each 
of  the  c&lculatlona  gives  a  total  chi  aquare  which  la  too  large  to  be 
conaistent  with  tha  residuals  being  drawn  from  a  random  sample. 

II1-5.  Comparison  of  Computing  Time.  Table  8  summariaea  the 
comparison  of  the  number  of  iterations  to  converge,  and  the  computing 
time  required.  As  seen  in  the  previous  discussions,  the  number  of 
iterations  was  usually  about  ths  same,  except  for  two  cases— the  cese 
4.1  under  the  Yew  Damper,  where  the  iteration  with  laaat  chi  aquare 
failed  to  properly  converge,  and  for  the  XM211  Pressure  Curves  trtiert 
the  ordinary  least  squarss  took  more  iterations  or  failed  to  converge 
As  seen  in  the  last  column,  for  all  the  cases  except  the  case  4.1 
under  the  Yaw  Damper,  tha  computing  time  is  comparable,  with  a  ten¬ 
dency  for  the  computing  time  to  be  longer  for  least  chi  square  than 
for  least  squares.  The  relative  difference  is  greater  whan  the 
original  total  computing  time  ia  short.  This  just  naans  that,  as 
would  be  expected,  it  takes  a  larger  fraction  of  the  computing  time 
to  compute  the  matrix  F  and  post  multiply  It  into  P*'  for  cases  where 

the  time  of  iteration  is  short. 

IV  CONCLUSIONS.  Based  on  four  different  types  of  non-linear 
theoretical  models  for  data  analysis,  our  results  indicate  that! 

(1)  Least  chi-squara  is  practicable  for  non-linear  analysis. 

(2)  The  computing  line  for  least  chi  square  la  longer  for  the 
models  which  use  less  computing  time,  but  because  the  convergence  of 
this  iterative  procedure  is  somewhat  better,  the  number  of  lterat.'one 
(and  particularly  the  number  of  "tries"  per  iteration)  is  raduced, 
thus  keeping  the  total  computing  time  about  the  same.  Models  with 
longer  integrating  time  would  expect  to  benefit  more  from  laaat  chi- 
square, 

(3)  With  one  exception  as  given  below,  the  least  chi-squara 
procedure  appears  to  be  leas  prone  to  failure  to  converge. 

(4)  When  the  autocorrelation  are  large  and  their  weighted  sum  is 
large  compared  to  the  chi-aquar«  for  the  residuals ,  the  iteration 
tends  to  produce  a  maximum  value  of  the  autocorrelations.  A  test  has 
been  devised  to  prevent  this  situation  from  occuring. 
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(At  the  time  of  final  editing  and  review  of  this  paper,  an  error 
was  diacovered  In  the  programming  of  the  calculation  of  the  variance 
of  the  autocorrelations,  Vj  •  The  error  amounte  to  only  a  few 
percent  but  would  make  it  difficult  to  reproduce  the  preaent  reaulta. 
Zt  ia  believed  that  the  main  thrust  of  the  reaulta  of  thia  paper 
remain  valid.) 
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TABLE  2.  Parameters  Fitting  Pressure  Curve 


ORDER 


1 

.717 

2 

.477 

3 

.286 

4 

.058 

5 

-.113 

6 

-.246 

7 

-.303 

8 

-.322 

9 

-.307 

10 

-.245 

11 

-.130 

12 

-.011 

13 

.069 

14 

.137 

15 

.110 

16 

.075 

17 

-.028 

18 

-.141 

19 

-.202 

20 

-.235 

xl?  57.9 

%)T2 


.680 

.696 

.427 

.443 

.247 

.248 

.027 

.013 

-.136 

-.  160 

-.258 

-.291 

-.315 

-.346 

-.334 

-.361 

-.312 

-.342 

-.240 

-.276 

-.104 

-.135 

.031 

-.035 

.130 

.048 

.217 

.123 

.203 

.106 

.168 

.085 

.037 

.0004 

-.068 

-.090 

-.149 

-.130 

-.205 

-.145 

62.96 

116.2 

33.01 

62.4 

95.97 

178.6 

Table  3.  Autocorrelations  and  Chi-Square  for  final  modal  of  XM211 
treasure  Oscillations. 

X22  based  on  the  first  5  Autocorrelations  for  Case  311-5, 
and  the  first  10,  for  case  511-10 


L(0,  4) 

L<4,  4) 

L<4,  2) 

L(4,  3) 

18.6 

53.1 

.605 

10.27 

17.70 

53.11 

1.094 

6.204 

18.58 

53.02 

0.605 

10.26 

R(I)  1 

.769 

-.111 

-.110 

2 

.431 

-.230 

-.232 

3 

.144 

.013 

.014 

4 

-.040 

.061 

.068 

5 

-.159 

-.124 

-.124 

6 

.029 

.030 

Sum  of  sqs 

.00275 

.03147 

.03138 

xi* 

.011 

28.90 

28.82 

xz ; 

24.78 

2.77 

2.49 

xT2 

24.79 

31.67 

31.31 

Slgnif 

-.959 

.273 

-.087 

.250 

(.033)2 

(.033)2 

Tabic  4,  Results  of  Yaw  Damper  Calculations 
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Differential  equations 


ITEM/ CASE 

KJE  1.0021-0 

KJE  1.0023' 

L(l,  2) 

.2155 

.2199 

U  2,  1) 

.4527 

.4442 

L(3,  1) 

.0431 

.0431 

L(4,  1) 

.0252 

.0251 

L(5,  1) 

.0743 

.0824 

R(l) 

.122 

-.086 

R<2) 

.065 

-.052 

R<3) 

.009 

.034 

R<4) 

.227 

-.223 

R(5) 

.179 

-.168 

*!* 

31.22 

31.24 

x22 

3.99 

3.04 

Xtot 

35.21 

34.28 

Significance  -•026 


-.086 


Table  6.  Results  of  Brookhaven  example  calculation.  Autocorrelation, 
X22,  Xt2  for  case  KJE  i. 0021-0  computed  for  comparison. 


ITEM/ CASE 

KWB  1.003-0 

KWB  1.005-3 

KWB  1.007-6 

Ldi.D 

5.318  X  10“4 

5.3183  X10"4 

5.262  X  10“4 

R<1) 

,782 

.782 

.786 

R(2) 

.44 

.44 

.453 

R(3) 

.098 

.098 

-.122 

R(4) 

-.204 

- 

-.170 

R(5) 

-.316 

- 

-.274 

R(6> 

-.235 

- 

.290 

121.14 

1.2  X  109 

121.89 

X2? 

36.71 

35.0 

xT2 

157.85 

156.89 

TABLE  7.  Results  of  Kinetic  Experiment  Water  Boiler  Calculations 
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Witt  v  ft-  *  ■ 

illlftnuikMML. 


■i  >o>-i 


ADJUST  AUTO-CORR  NO  OF 


CASE 

PARAM 

RANK 

ITER 

TIME  (SEC) 

KEWB  KINETICS 

1.003 

1 

0 

4 

84. 

1.005-3 

1 

3 

4 

81. 

1.007-6 

1 

6 

4 

89. 

YAW  DAMPER 

1-6 

4 

0 

3 

23.6 

4 

3 

0 

6 

11.2 

4.1 

3 

5,6,12 

6 

11.2  (FAILED) 

4.011 

4 

6 

2 

8.5 

4.012 

4 

0 

4 

8.2 

BROOKHAVEN 

1.0021 

5 

0 

5* 

14.9 

1.0023-5 

5 

5 

7 

21.5  (15.2) 

XM211  PRESSURE 

BCK 

6 

0 

14 

6.5 

BGK1.101 

6 

5 

11 

11. 

3.002 

6 

0 

7 

11. 31-/ FAILED 

3.102-5. 

6 

5 

15 

23. 

3.102-5 

6 

5 

11* 

18.2 

TABLE  8.  Comparison  of  Computing  Time. 
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Figure  4  -  Fit  to  Full  Data  of  Pressure  Curve  of  ?M211  Propellant  Charge  by 
Least  Chi  Squares. 
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ON  THE  POWER  OF  BIRNBAUM'S  TEST 


Ray  E,  Schafer 

Hughe*  Aircraft  Co.,  Fullerton,  CA  02634 


Z,  W.  Btrnbaum  has  proposed  a  hypothesis  test  procedure  which,  under  fairly 
general  conditions,  does  not  require  explioit  knowledge  of  the  oritical  values 
of  the  test  statietio.  In  this  paper  we  investigate  the  power  of  the  test  in  a 
variety  of  situations.  In  particular  we  have  considered  situations  in  whioh  the 
underlying  observations  have  normal  and  chi-square  related  distributions.  We 
show  that  the  asymptotic  power  of  this  test  is  identical  to  the  olassloal  test  using 
the  same  statistic  and  that  the  Bimbaum  test  achieves  its  asymptotic  power  very 
rapidly. 

The  normal  case  is  considered  both  for  complete  and  oensored  samples. 

1.0  INTRODUCTION 

The  olassloal  hypothesis  testing  problem  Involves  the  sampling  distribution  of 
the  test  statistic  (say  S).  For  example,  to  test 

Hgi  DF  (distribution  function)  is  N  (pQ,  1) 

versus 

Hj :  DF  is  N(/i  1)  ^  >  Uq 


o  o 

whore  N(p,  a  )  means  normal  with  mean  n  and  varianoe  cr  , 

the  test  statistic  is  S  ■  X,  the  sample  mean,  and  the  CRITICAL  VALUE,  say  S*, 

is 

8*  «  X*  -  M0  +  Here  Z^is  the  (1  -  a) 

*  * 

quantile  of  the  standard  normal  distribution  and  n  is  the  random  sample  size. 

The  sampling  distribution  qf  S  -  X  is  well  tabled  so  that  it  is  easy  to  obtain 
S*  ■  X*  the  critioal  value, 

In  many  situations  however  the  sampling  distribution  of  S  Is  analytically 
intractable.  For  example  consider  the  Weibull  DF: 

Fx(x)  «  1  -  oxp  [p  (x/b)^j  ,  b,  6,  x  >  0, 

*  0  elsewhere » 

The  sampling  of  the  maximum  likelihood  estimate  of  o,  say  6,  is  intractable. 
However,  the  sampling  distribution  of  S/o,  while  intraotable  has  a  distribution 
free  of  b  and  c.  Thus,  the  DF  of  c/o  oould  be  obtained  (Indeed  was  obtained  by 
Thoman,  Bain  and  Antle,  1989,  Technometrics  11,  445-400)  by  Monte  Carlo 
methods.  The  Monte  Carlo  approach  is  quite  expensive;  involving  some  10,000 
to  50,000  c/cfs  for  each  n, 

Z,  W,  Blmbaum  ("Computers  and  Unconventional  Test  Statistics,"  1974, 
Reliability  and  Biometry,  Eds.  j  F.  Proschan  and  R.  J,  Serfling,  SIAM ,  441-458 
and  "Testing  for  Intervals  of  Increased  Mortability, "  1975,  Reliability  and  Fault 
Free  Analysis,  Eds,:  Richard  E.  Barlow,  Jerry  B,  Fusseii  and  Nozer,  D. 


Singpurwalla,  SIAM  413-426)  has  proposed  a  remarkedly  simple  procedure  In  the 
situation  of  testing 


Hq!  DF  of  S  ■  Gq(s) 

versus 

DF  of  S  ■  G^(s) 

where  O  Is  analytically  Intractable.  Birnbaums  test  avoids  the  expense  of  a  large 
Monte  Carlo  simulation.  Here  we  will  investigate  the  power  of  the  Birnbaum 
test  (B.  T. )  in  a  variety  of  situations. 

2.0  B.  T.  DESCRIPTION 

We  disouss  in  this  seotion  a  right  (upper)  tail  hypothesis  test.  Obvious  modifica¬ 
tions  lead  to  left-tail  and  two-tail  situations. 

Let  F(x,0)  be  the  distribution  funotion  for  random  variable  X  and  let  S  be  a 

.  n 

test  statistic  for  0,  based  on  a  sample  of  size  n.  Suppose  G0(sn)  and  Gi(®n) 
are  the  distribution  functions  for  Sn  when  0»  0Q  and  0-  6^  respectively.  The 
B.T.  requires  that,  for  all  real  Sn  «n)  5  GQ(y.  and  for  at  least  one  SR 
Gi(Sn)^G0(  Suppose  that  a  random  sample  of  n  observations,  x1,  . , , ,  xQ, 
is  available  whioh  has  been  used  to  calculate  a  single  value  of  8n  say  s* ,  and 
we  wish  to  choose  between  H^j0  *0^  and  H^i  0»  0^  on  the  basis  of  this  observation. 

Birnbaum  has  shown  that,  if  it  is  possible  to  obtain  a  random  sequence  of  N 
observations  of  Sn  using  only  F(x,  9q),  a  hypothesis  test  may  be  performed  by 
selecting  a  number,  y  0  <  y  <  1,  and  observing  the  number,  M,  of  these  N 
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observations  whioh  are  greater  than  s*  with  rejection  of  HQ  if  and  only  if  M/N  2  y . 
Birnbaum  shows  that  the  size  of  his  test  is 


O  -  (1  +  [NYJ(l  +  N) 
where  £ny]  is  the  greatest  integer  *  NY, 

In  practice,  one  performs  a  B.  T,  of  exact  size  a  as  follows.  Choose  0  <  Y  «f„l, 
the  size  desired,  and  choose  an  integer  N  suoh  that  (N  +  1)  is  a  positive  Integer. 
Generate  N  observations  of  Sn  under  HQ  and  observe  M/N,  Reject  HQ  if  M/N 

2  (a(N  +  1)  -  i)/N  ■  Y, 

3.0  CASES  INVESTIGATED 

The  advantage  of  the  B.T,  is  that  a  knowledge  of  C0(sn),  the  sampling  distribution 
of  SR  under  HQ  is  not  required.  The  N  values  of  Sn  may  be  generated  by  Monte 
Carlo  methods  directly  from  F(x,  0q).  The  oost  of  the  E.  T.  is  a  funotion  of  both 
n  and  N,  and  may  be  substantial  (although  muoh  cheaper  than  a  "full"  Monte 
Carlo  simulation  of  oaloulation  of  sn  must  be  performed  by  iterative 

methods.  Hence,  it  is  Important  to  know  how  the  power  of  the  B.T.  varies  with 
n  and  N. 

We  have  investigated  the  power  of  the  B.T.  for  test  sizes  a  -  0.01,  0,05  and 
0,10:  for  sample  sizes  n  ■  5,  10,  20,  and  50  and  for  N  ■  9,  19,  39,  99,  199  and 
499,  It  should  be  noted  that,  fora  -  0,01,  no  B.T.  exists  with  N»  9,  19,  39  and 
for  a  -  0. 05,  no  B,  T,  exists  with  N  ■*  9,  In  fact,  no  B,  T.  of  size  a  can  be  obtained 
unless  (N+  l)"1*^. 
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We  have  restricted  our  investigation  to  maximum  likelihood  estimators  (or 
functions  thereof)  and  to  the  following  distributions  and  parameters. 

a)  Normal  distribution  -  tests  for  the  mean,  0  (known  o),  with  complete 
samples,  and  tests  for  the  mean  (unknown  o)  with  Type  II  censored  samples, 

b)  The  general  olass  of  tests  for  which  Sn  has  a  chi-square  distribution 
This  inoludes  tests  for  a  (known  or  unknown  0)  in  the  normal  distribution  and 
tests  for  the  mean  of  a  one-parameter  exponential  distribution. 


4.0  POWER  OF  THE  BIRNBAUM  TEST 

Intuitively,  It  is  clear  that  letting  N-*  *  is  tantamount  to  obtaining  the  exact  sam¬ 


pling  distribution  of  SQ:  hence,  the  asymptotic  power  with  respeot  to  N  should  be 
Identical  to  the  classical  power  based  on  the  same  statistio. 


The  B.  T.  power  under  Hj  is 


1 -^(N)-P(M/N^f|H1)  . 

-j-M  N 

■  JE  (Jt1- w)aoc/')doi<v- 

fl  [NV] 

'Jo  § 


where  J1(u)  -  G1(G~1(u)),  Blmbaum  has  proven  that 
lim(l  -  0(N))*»  1  -  Jx(l  -?). 

N-« 


A  statistician  is  seldom  interested,  in  knowing  the  power  of  a  test  to  even  three 
docimal  places.  Clearly,  the  requirement  of  extreme  aoouraoy  in  power  deter¬ 
mination  increases  the  ooat  of  computation.  We  have  ohosen  to  relax  the  aoouraoy 
requirement  so  that  more  distributions  and  sample  sizes  could  be  studied. 

All  of  the  results  were  obtained  on  a  CDC  Cyber-173  computer.  Where  power  was 
obtained  by  numerical  integration,  the  trapezoid  rule  was  used  with  200  equal 
Intervals  over  the  domain  of  Integration.  Where  the  limits  of  integration  extended 
to  A «>,  the  heuristic  limits  used  were  the  0, 0001  and  0. 9909  quantiles  (A3. 895,  for 
example,  with  the  normal  distribution).  As  a  check  on  the  numerical  aoouraoy  of 
the  integrations,  both  "tails"  were  evaluated.  That  is,  we  determined  1  -  p  and 
p  separately.  In  every  case,  the  sum  of  the  two  was  in  the  domain  (0. 995,  1. 005). 

Where  Monte  Carlo  methods  were  employed,  the  random  number  generating 

algorithm  was  the  multiplicative  congruential  method  suggested  by  Knuth 
48 

using  modulo  2  arguments,  For  the  Monte  Carlo  simulatlonst 

a)  If  it  was  neoessary  to  determine  olassioal  power  by  simulation,  10, 000 
observations  of  Sn  when  0-0q  were  obtained  and  utilized  to  estimate  the  0, 90,  0. 95 
and  0. 99  quantiles  of  the  distribution  under  HQ»  Then  10, 000  observations  of  Sn 
when  0«0j  wore  generated  and  compared  with  these  quantiles. 


b)  To  determine  the  power  of  the  B.T.  by  simulation,  a  single  observation  of 
$n  (9*9^)  was  obtained  and  oompared  with  N  observations  of  Sn  (9=9q).  The  number 
M  of  these  N  observations  greater  than  Sn(8=61)  was  recorded,  and  if  M  N  £ 

(a(N  +  1)  -  1)/N,  the  null  hypothesis  was  rejeoted.  This  complete  procedure  was 
repeated  2500  times. 

It  appears  that  a  B.  T.  using  N-199  or  greater  could  substitute  for  a  olassioal 

« 

tost  on  the  mean  of  a  normal  distribution  with  virtually  no  loss  in  power.  This 
may  give  a  praotioing  statician  some  oonfidenoe  in  using  the  B.T.  for  problems 
where  the  distribution  of  Sn  is  not  obtainable,  or  obtainable  only  at  great  expense. 

1  • 

6.0  NORMAL  MEANS  (KNOWN a ) 

Of  course,  no  one  would  ever  use  a  B.  T.  in  place  of  a  Neyman-Pearson  test  for  a 
hypothesis  about  the  mean  of  a  normal  distribution  with  known  o .  The  p.  d.  f.  of 
Sn  <1.  e. ,  the  sample  mean)  under  both  the  null  and  alternative  hypotheses  and 
henoe  the  power,  Is  known  analytically.  But  such  an  artificial  case  is  valuable 
for  studying  the  B.T.  for  precisely  this  reason.  We  may  observe  the  relative 
power  of  the  B.T.  in  comparison  to  the  olassioal  test  as  a  Amotion  of  N,  n,  and 
the  classical  power,  to  get  a  "feel"  for  the  behavior  of  the  B.T.  as  a  function  of 
sample  size. 

Let  X  bo  N(8,  o'2).  W*  obose  as  the  null  hypothesis  N(0,  1)  and  as  alternatives 
d^.l,  0.2,  0.5,  1.0,  and  2.0.  The  power  of  the  B.T.  was  obtained  analytically 
by  numerical  integration. 


The  efficiency  of  the  B.  T.  Relative  to  the  classical  test  was  remarkably  high,  even 
for  small  N.  Over  the  entire  domain  of  samples  sizes,  n,  the  B.T.  power  was 
never  less  than  85%  of  the  olassioal  power.  For  N  -  1B9  and  499,  the  B.T,  power 
was  never  less  than  95%  of  the  olassioal  power.  As  an  example,  for  N  »  199,  we 
obtained} 


n  -  5 


B.T.  Power 
Olassioal  Power 
B.T.  Power 
Olassioal  Power 


^0,5  5*1.0 

0.29  0.71 

0.300  0.723 
n  *  20 

0.71  0.99 

0.723  0.998 


n  ■  10 

5-0.6  0*  1,0 

0.46  0.93 

0.475  0.935 
n  ■  50 

0.97  1.000 

0.971  1.000 


7.0  NORMAL  MEANS  {UNKNOWN  a)  WfTH  CENSORED  SAMPLES 
The  olassioal  power  for  hypothesis  tests  on  normal  means  with  the  standard 
deviation  unknown,  but  constant,  is  available  for  complete  random  samples 
through  tables  of  the  non-central  t-dlstribution.  Here,  we  examine  Type  II  cen- 
sored  samples,  where  no  suoh  power  distributions  are  available.  Given  a  cen¬ 
sored  sample. 


X(l)’ X<2)'  X(r)  r<n. 

to  _  2 

the  sample  mean,  >?  and  standard  deviation  s  -  La  (x^  -  x')  /r  are  calculated. 
An  auxiliary  funotlon  X  is  needed.  The  value  of  X  depends  only  on  r/n  and  on 
i,2/(x’  -  x^)  2.  The  M.  L,  E,  for  5  is  x*  -  X{x»  -  x^). 


1 9b 


Hypothesis  tests  of  the  form  HQt  9  *9q  vs,  H1se>eQ  were  examined  for  the  oases 
(0j_  -  9^/<fm  0.1,  0,2,  0.6,  1.0  with  sample  sizes  5,  i0,  20,  50  an U  censoring 
at  0, 8n  for  n  ■  5,  10,  20  send  at  0,4n  for  n  *  10,  20,  50.  Both  the  classical  power 
and  B.T.  power  were  obtained  by  Monte  Carlo  simulation,  as  described  previously. 


The  B.T.  with  N  -  199  or  499  will  provide  essential iy  equivalent  to  that  of  the 
classical  test  for  all  of  the  parameters  and  sample  sizes  examined. 


8.0  THE  CHI-SQUARE  CASE 

o 

Many  hypothesis  testing  situations  Involve  test  statistics  whioh  have  a  X  distri¬ 
bution.  e.  g. ,  tests  on  the  variance  of  a  normal  distribution  (known  or  unknown 
mean)  and  tests  on  the  mean  of  an  exponential  distribution.  The  power  of  the 
olassioal  X  2  test  Is  available  in  the  literature  from  tables  of  the  non-central 

n 

X  distribution,  But  we  have  explored  this  oase  for  the  same  reason  that  the 
normal  distribution  was  examined  -  the  B.  T,  power  may  bo  obtained  analytically 
and  its  behavior  with  respect  to  N  may  lend  oredenoo  to  the  assertion  that  the 
B.T.  Is  essentially  as  powerful  as  the  classLoal  test  for  a  variety  of  probability 
distributions. 


Many  hypothesis  tests  involving  the  X‘ 


"o'  s„  1  xl 
"r  4  Xm 


distribution  are  equivalent  to 


where  Sn  is  the  test  statistic,  m  is  degrees  0f  freedom  and  0<a<l  for  a  right- 
tail  test,  a  >  1  for  a  left-tall  test.  We  examined  right-tail  tests  for  m  -  5,  10, 

20  and  for  a«2/3,  1/2,  1/3,  l/4,  a  “0.10,  0,05,  0,01. 
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The  B.T,  power  was  remarkably  close  to  the  classical  power,  even  for  N  -  9. 

In  fact,  the  smallest  ratio  of  B.  T.  power  to  olassioal  power  for  all  the  combina¬ 
tions  investigated  was  0. 87.  A  sample  of  results  for  N  *  199  and  «  «  0. 05  1st 


m  ■  6 

m  ■ 

10 

a  1/4 

1/3 

1/2 

2/3 

1/4 

1/3 

1/2 

2/8 

B.T.  Power  0.72 

0,59 

0.34 

0.19 

0.91 

0.80 

0.51 

0.27 

Classical  Power  0.736 

0.595 

0.354 

0. 194 

0.918 

0. 807 

0.518 

0.272 

m  ■  20 

a  1/4 

1/3 

1/2 

2/3 

B.T.  Power  0.99 

0.96 

0.73 

0.39 

Classical  Power  0.993 

0.960 

0.736 

0.401 

2 

Hence,  the  t  data  support  earlier  conclusions  that  the  B.  T.  with  a  reasonable 
value  for  N,  say  199,  is  essentially  as  powerful  as  a  olassioal  test. 

9.0  CONCLUSIONS 

We  have  investigated  the  power  of  the  B.  T.  with  respect  io  the  power  of  the  cor¬ 
responding  classical  test  in  a  variety  of  situations.  These  situations  Included 
complete  and  Type  II  censored  samples  forthe  commonly  used  test  sizes  and 
frequently  used  sample  sizes. 

It  seems  clear  that  the  B.  T.  offers  cost  savings  when  the  sampling  distribution  of 
the  test  statistic  is  unknown  and  must  be  obtained  by  expensive  methods. 


In  addition  to  the  fact  that  the  asymptotic  power  of  the  B.  T. ,  as  N-*  «  ,  is  equal 
to  the  power  of  the  classical  test  based  on  the  same  statistic,  the  B.  T.  has  some 
interesting  characteristics.  In  all  cases  the  relative  power  of  the  B.T.  was  quite 
large  even  for  N  as  small  as  9  and  generally  for  N  2199  the  power  was  95%  of 
the  asymptotic  power  or  greater.  Also,  generally,  the  relative  power  of  the  B.T. 
increased  as  the  alternate  hypothesis  got  further  away* from  the  null  hypothesis. 
Finally  the  B.T.  relative  power  Increased  with  N. 
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ERROR-TIME  RESPONSE  PERFORMANCE  OF  NAIVE  SUBJECTS 


Michael  Hacskaylo  and  Joseph  E.  Swlstak  ^ 

Night  Vision  and  Electro-Optics  Laboratory  J 

Fort  Belvolr,  Virginia  22060  ] 


Abstract.  An  experiment  was  conducted  In  which  the  error  and  time 
response  performance  for  designating  the  angular  location  of  a  single 
flash  of  light  on  a  circular  pattern  of  lights  were  measured.  Fourteen 
naive  subjects  were  Instructed  to  record  as  accurately  and  rapidly  as 
possible  the  angular  position  of  an  activated  light.  They  were  allowed 
only  one  attempt  for  each  of  six  consecutive  trials.  The  data  are 
presented  In  terms  of  mean  time  of  each  response  per  trial  and  mean 
error  per  trial.  The  mean  error,  as  a  function  of  mean  time,  appears 
to  be  bounded  by  an  error-time  response  equation;  E  ■  -20.88  log(t/15.37) , 
where  E  Is  the  mean  angular  error  In  degrees  and  t  Is  the  mean  time  In 
seconds.  Surprisingly,  the  subjects  responses  as  measured  In  either 
time  or  error  did  not  follow  classical  reaction  time  or  learning  patterns. 
That  Is,  while  time  of  response  remained  fairly  consistent  from  trial 
to  trial,  the  lowest  error  occurred  on  the  first  trial  while  maximum 
error  consistently  occurred  on  the  fourth  trial.  Based  upon  the  six 
trial  limit  used  in  the  experiment,  It  Is  believed  that  the  naive 
subjects,  first  trial  performance  Is  the  best  for  designating  the 
angular  location  of  a  single  flash  of  light. 


Introduction.  The  philosophy  which  tank  crewmen  have  always  adopted 
has  been  "make  your  first  shot  count  because  you  may  not  get  a  second 
chance."  This  philosophy  has  become  more  acute  with  the  recent  advent 
of  "SMART"  weapons  which  ride  beams  of  light  to  a  target.  A  system  was 
designed  which  would  allow  tank  crewmen  to  detect  and  radially  demarcate 
the  source  of  designation  by  a  coherent  light  source.  The  system  Is 
designed  to  operate  by  having  a  tank  crewman  observe  a  circular  array 
of  lights  on  a  panel.  When  the  tank  Is  Illuminated  by  a  laser  beam, 
a  corresponding  azimuthal  light  Is  activated.  The  crewman  would  Interpret 
and  record  the  azimuthal  position  for  appropriate  tank  action.  The 
effectiveness  of  the  crew  would  depend  upon  (I)  the  speed  and  accuracy 
with  which  the  azimuth  is  read  out,  and  (2)  the  panel  configuration  used 
to  d'splay  the  azimuthal  Information.  The  panel  used  In  this  experiment 
was  designed  from  a  technical  consideration  based  on  the  circular 
representation  of  equally  spaced  light  bulbs  (Fitts  and  Seeger,  1953). 


Method. 

Subjects.  Fourteen  U.  S.  Army  enlisted  men  of  various  ranks  were 
randomly  selected  from  a  large  group  of  Individuals  to  serve  as  subjects. 
None  had  prior  training  In  tanks  or  tank  related  equipment  and  none  had 
prior  experience  with  the  display  panel  being  tested.  The  fourteen  subjects 
were  then  randomly  assigned  to  one  of  two  groups  comprised  of  seven  subjects 
each. 


Apparatus.  The  stimulus  display  panel  consisted  of  a  9cm  diameter 
ring  of  36  equally  spaced  light  bulbs.  This  panel,  portrayed  In  Figure  1, 
was  positioned  on  the  front  panel  of  a  box  20cm  long,  10cm  high  and  5cm 
deep.  The  light  bulbs  were  angularly  marked  from  zero  to  360  In  ten 
degree  Increments  In  a  clockwise  direction  with  zero  at  the  top. 

Responses  were  recorded  on  a  response  panel.  This  consisted  of  a 
12cm  circle  drawn  on  a  20x25cm  sheet  of  plain  paper.  The  circle  was 
divided  Into  quadrants  and  marked  Into  degrees  as  follows}  Zero  degrees 
(0°)  was  marked  at  the  top.  In  a  clockwise  direction,  each  quadrant 
was  successively  marked  90°,  180®,  270®,  and  again  at  the  top,  360®. 

A  pencil  was  used  for  marking  angular  positions  with  an  "X"  on  the  circle. 

Procedure.  Each  subject  was  briefed  Individually  prior  to  his 
participation  In  the  experiment.  They  were  brought  Into  a  room  which 
contained  the  stimulus  display  panel,  a  bench,  chair  and  associated 
equipment  required  to  activate  the  lights  of  the  panel.  Each  subject 
was  briefed  as  follows t 

"As  accurately  and  as  rapidly  as  possible,  determine  the  angular 
location  of  a  light  when  It  comes  on  and  mark  with  an  "X",  that  position 
on  the  circle  on  the  sheet  of  paper  In  front  of  you.  The  sheet  of  paper  was 
referred  to  as  the  response  panel  for  purposes  of  the  study.  Each  subject 
was  allowed  two  faml larlzatlon  trials  to  ba  sura  they  understood  the 
Instructions.  Each  subject  was  than  given  six  trials.  The  sequence  of 
lights  for  trials  one  to  six  are  presented  In  Figure  2. 


FIGURE  1.  2-0  REPRESENTATION  OF  STIMULUS 
PANEL  COMPRISED  OF  36  LIGHTS 
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Etch  aubject  saw  this  same  sequence .  A  new  response  panel  was  supplied 
for  each  trial.  The  time  Interval  from  when  the  light  came  on  to  when 
the  subject  marked  the  panel  was  measured  by  a  stop  watch  to  0.01  seconds. 

The  stop  watch  was  controlled  by*  the  experimenter  and  It  was  assumed 

that  the  reaction  time  error  Introduced  was  fairly  constant. 

Upon  completion  of  a  set  of  six  trials,  the  subject  was  dismissed. 

The  subjects  tested  versus  those  not  tested  were  kept  In  separate  rooms 

until  all  seven  subjects  In  a  group  were  finished.  One  group  (A)  of 
seven  subjects  was  tested  on  one  day,  the  other  group  (B)  of  seven 
subjects  was  tested  on  the  following  day. 

The  angular  positions  marked  on  the  response  panels  were  scored 
tn  degrees  by  using  a  transparent  template  graduated  to  0.5  degrees  and 
superimposed  on  the  marked  response  panel.  The  accuracy  of  the  marked 
position  was  then  measured  to  +  0.5  degrees  which  was  the  resolution 
of  the  scoring  template. 

Results  and  Discussion. 

The  mean  time  of  response  for  each  trial  are  presented  In  Figure  3* 


FIGURE  3.  MEAN  RESPONSE  TIME  TO  DESIGNATE  ANGULAR  LOCATION 
OF  A  FLASH  OF  LIGHT  ON  SIX  CONSECUTIVE  TRIAL! 


These  mean  times  and  associated  standard  deviations  are  presented  In 
Table  1.  In  general,  It  appears  that  the  mean  time  to  respond  did  not 
vary  significantly  over  the  six  trials.  There  does  appear  to  be  a  gradual 
decrease  In  response  time  from  trial  one  to  trial  five,  but  an  Increase 
on  trial  six.  There  was  no  readily  obvious  reason  for  this  Increase 
on  the  sixth  trial,  t.e.,  no  subject  took  an  Inordinate  amount  of  time 
It  would  also  seem  that  fatlque  could  not  be  a  factor  with  only  six 
trials  having  elapsed. 


TRIAL  NO. 

1 

BDDDD 

MEAN  TIME  (SEC) 

3.79 

3.94 

3.21 

3.34 

B3 

342 

STANDARD  DEVIATION 

1.41 

2.03 

m 

1.49 

1.00 

1.23 

TABLE  1.  MEAN  TIME  AND  RELATED  STANDARD  DEVIATION  REQUIRED 
TO  DETECT  AND  DESIGNATE  THE  ANGULAR  LOCATION  OF  A 
LIGHT  FLASHED  WITHIN  A  360°  ARRAY  OF  LIGHTS  ON  SIX 
CONSECUTIVE  TRIALS. 


then  Increased  with  the  greatest  error  occurring  on  the  fourth  trial, 
after  which,  the  error  decreased.  The  error  on  the  sixth  trial  was 
very  near  the  error  on  the  first  trial.  A  large  part  of  the  error 
on  trial  four  can  be  attributed  to  the  performance  of  two  subjects  who  had 


TABLE  L  MEAN  ERROR  AND  RELATED  STANDARD  DEVIATION  ASSOCIATED 
WITH  THE  ANGULAR  DETECTION  AND  DESIGNATION  OF  A  LIGHT 
FLASHED  WITHIN  A  360*  ARRAY  OF  LIGHTS  ON  SIX  CONSECUTIVE 


errors  of  94.5  and  68.0  degrees,  on  that  trial.  However,  even  with  these 
two  values  removed  from  the  data,  the  mean  degrees  of  error  for  trial 
four  remains  at  16.2.  If  this  level  of  error  Is  the  more  accurate,  then 
It  can  be  said  that  the  third  and  fourth  trials  wars  the  worst  In  terms 
of  performance,  and  the  dotted  portion  of  Figure  4  would  more  aptly  ' 
represent  the  performance  on  this  task.  The  mean  error,  as  a  function 
of  mean  time,  (Figure  5)  appears  to  be  bounded  by  an  error-time  response 
equation:  E  —20.88  1og(t/15.37) »  where  E  Is  the  mean  time  In  seconds.. 
This  curve  provides  somewhat  of  an  upward  estimate  of  angular  error  given 
an  elapsed  period  of  time  for  a  response  -  the  greater  the  time,  the  lower 
the  error. 


However,  regardless  of  which  curve  In  Figure  4  most  eccuratety 
represent  the  performance  which  could  be  expected  on  the  task  described 
In  this  study,  It  Is  Interesting  to  note  that  neither  set  of  date 
follows  the  expected  learning  pattern  described  In  learning  literature. 
Normelly,  one  could  expect  accuracy  to  be  poorest  on  the  first  trial, 
rapidly  Improve  on  the  next  few  trials  and  then  continue  to  Improve 
at  a  slower  rate  until  some  maximum  level  was  reached.  The  number  of 
trials  required  for  asymptotic  performance  to  occur  would  depend  upon 
the  degree  of  difficulty  of  the  task.  The  present  task  should  have  , 
required  6-8  trials.  It  appears  that  asymptotic  performance  was  being 
approached  on  the  stkth  trial,  but  whet  Is  truly  Interesting  Is  that 
the  performance  on  the  first  trial  was  actually  better  then  on  the  sixth. 

In  terms  of  an  untrained  subject  being  able  to  determine  angular 
direction  of  designation,  the  first  attempt  he  made  would  be  the  most 
accurate  of  his  Initial  six  attempts. 

The  conclusion  of  this  study  must  be  that  the  Initial  attempt 
by  an  untrained  gunner  would  be  at  least  as  accurate  as  one  who  Is 
starting  to  asymptote. 
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IMAGE  INTERPRETATION  PERFORMANCE 
ON  FOUR  STANDARD  TYPES  OF  AEROGRAPHIC  FILM 
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ABSTRACT 


This  study  Involved  trained  operational  Image  Interpreters  who 
analyzed  highly  controlled  aerial  Imagery  from  which  the  effects  of 
type  of  film  upon  target  detection  were  determined.  One-hundred-and- 
one  operational  Image  Interpreters  generated  the  following  mean  target 
detection  probabilities:  Color  Infrared  -  58. 6%,  Color  -  55. 4%, 
Panchromatic  -  44. 7%,  and  Black  and  White  Infrared  -  43. 4*.  At  the 
0.05  significance  level,  target  detections  were  affected  by  film  type 
as  follows:  Color  Infrared  differed  significantly  from  both  Panchro¬ 
matic  and  Black  and  White  Infrared  films.  Color  differed  from  Black 
and  White  Infrared.  The  combined  mean  of  target  detection  for  Color 
Infrared  and  Color  differed  significantly  (0.01  level)  from  the  combined 
mean  for  Panchromatic  and  Black  and  White  Infrared.  Therefore,  use  of 
Color  and  Color  Infrared  Imagery  results  Is  significantly  more  accurate 
day  Image  Interpretation. 


1.0  INTRODUCTION 


The  purpose  of  this  study  was  to  quantitatively  determine  the  target 
acquisition  capabilities  by  Image  Interpreters  (II)  of  the  following  film 
types:  Aerochrome  MS  2448  (Color),  Aerochrome  Infrared  2443  (Color  Infrared), 
Plus  X  Panchromatic  2042  (Black  and  White),  and  Infrared  Aerographlc  2424 
(Black  and  White  Infrared).  Image  Interpretation  Is  defined  1/  as  the 
examination  of  Images  of  objects  on  film  for  the  purpose  of  Identifying 
the  objects  and  deducing  their  significance. 

Approximately  90  percent  of  the  Intelligence  gathered  In  World  War  II 
was  derived  from  aerial  photography.  The  requirement  for  accurate  Imagery 
Intelligence  Is  escalating  as  weapon  systems  and  tactics  develop  and  become 
more  refined.  To  obtain  this  Information,  It  Is  becoming  Increasingly  more 
common  to  use  color,  color  Infrared,  and  black  and  white  Infrared  film. 
Strandberg  2/  states  "  color  aerial  photography  offers  much  promise  In  the 
gathering  of  Imagery  Intelligence,  because  humans  have  the  capability  of  dis¬ 
criminating  between  an  almost  Infinite  number  of  different  colors,  but  at  most, 
only  a  few  hundred  different  shades  of  gray". 

2.0  TEST  SITE 

An  820  acre  site  was  selected.  This  site  Is  used  for  equipment 
evaluation  by  the  US  Army.  Military  equipment  and  camouflage  devices 
such  as  nets  were  randomly  located  throughout  the  study  area.  The 
soil  contained  a  high  moisture  content  and  the  color  was  reddish-tan.  The 
brush  was  gray  and  brown  In  color.  Included  within  this  site  were  building 
complexes,  open  fields,  dense  woods,  and  clumps  of  green  grass.  The 
forest  composition  was  a  mixture  of  oak  and  pine. 

3.0  TEST  IMAGERY 

Photographic.  Images  consisted  of  a  13  frame  series  of  9"  X  9"  positives 
taken  with  60%  forward  overlap.  One  frame  series  was  acquired  for  each 
of  the  four  standard  types  of  aerial  film.  Film  and  filter  characteristics 
are  summarized  In  Table  1. 
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_ FILM  CHARACTERISTICS 


RESOLVING  SPECTRAL  RESULTANT 

POWER  (T.O.C.)*  SENSITIVITY  FILTER  SPECTRAL 

FILM  TYPE  1000;!  1.6:1  RANGE  (NM)  CUT-OFF  SENSITIVE  (NM) 


Plus  X 

Panchromatic 

2042 

(Black  & 
White) 

100 

50 

250-700 

Zeiss  Yellow 
490  NM 

490-700 

Infrared 
Aerographlc 
2424 
(Black 
&  White) 

80 

40 

400-900 

Zeiss  Orange 
550  NM 

550-900 

Aerochrome 

MS  2448 
(Color) 

80 

40 

400-700 

Zeiss  Clear 
Actlvlg 

No  cut  off 

400-700 

Aerochrome 

Infrared 

2443 

(Color) 

S3 

32 

400-900 

Zeiss  Orange 
550-NM 

550-900 

"Target  Object  Contract 


A  KC-4B  camera  system  with  a  6  Inch  focal  length  lens  was  used.  In  all 
cases  except  black  and  white  Infrared  film,  standard  f 1 1m/fl 1  ter  combinations 
were  employed.  A  zelss  orange  filter  was  used  with  the  black  and  white 
Infrared  film  Instead  of  a  red  filter;  to  Increase  the  spectral  response, 
and  therefore,  the  Information  content  of  this  film  type.  All  Imagery  was 
gathered  during  four  overflights  (one  per  film  type)  at  an  altitude  of  1500 
feet  above  ground  between  the  hours  of  1100  and  1400.  Therefore,  the  sun 
angle  effect  was  negligible.  The  photographs  wore  taken  In  February,  and  the 
weather  was  clear  and  sunny.  Each  9  Inch  photograph  covered  a  land  area 
of  approximately  124  acres.  The  total  number  of  targets  present  or  detected 
for  each  strip  of  Imagery  were  determined  by  three  senior  image  Interpreters. 
They  performed  detailed  and  exhaustive  analysis  upon  the  Imagery.  The 
results  of  which  are  presented  In  Table  2. 
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TABLE  2 

TOTAL  MILITARY  TARGETS  DETECTED  FOR  EACH  TYPE  OF  FILM 


FILM  TYPE 

NUMBER  OF  TARGETS 

Plus  X  Panchromatic  2042 

(Black  and  White) 

46 

Infrared  Aerograph 1c  2424 
(Black  and  White) 

50 

Aeochrome  MS  2448 
(Col  or) 

44 

Aerochrome  Infrared  2443 
(Color) 

47 

Variations  In  the  number  of  targets  detected  between  film  types 
(ground  truth)  are  not  significant.  They  were  apparently  due  to  slightly 
different  flight  lines  flown  by  the  photgraphlc  aircraft. 

4.0  METHODS  OF  PROCEDURE 

The  Pseudo- Isochromatlc  Plates  for  Testing  Color  Perception,  developed 
by  the  American  Optical  Corporation,  were  given  to  each  II  In  order  to 
Insure  that  the  Interpreters  were  not  color  deficient.  A  total  of  101 
operational  US  Marine  Corps  Image  Interpreters  participated  In  this  study. 
The  participants  were  randomly  divided  into  four  groups,  one  group  for 
each  type  of  film.  The  assumption  was  made  that  the  four  groups,  due  to 
the  Central  Limit  Theorem,  contained  Interpreters  of  equal  ability.  Each 
II  was  Instructed  to  perform  detailed  Image  analysis  to  detect  military 
targets  such  as  jeeps,  trucks,  etc.,  and  was  allotted  46  minutes  to  analyze 
a  selected  film  strip.  Each  II  viewed  only  one  strip  of  film,  and 
consequently  only  one  film  type. 

5.0  RESULTS 

The  percentage  of  military  targets  detected  by  the  Image  Interpreters 
for  each  of  the  four  types  of  film  was  calculated.  These  data,  along 
with  the  associated  standard  deviations,  95%  confidence  Intervals,  and 
sample  sizes  are  presented  in  Table  3. 
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TABLE  3 


STATISTICAL  SUMMARY  FOR  MEAN  PERCENTAGE  OF  TARGETS  DETECTED  BY  FILM  TYPE 


FILM  TYPE  MEAN  PERCENTAGE  STANDARD  95*  CONFIDENCE  SAMPLE  SIZE 

OF  TARGETS  DEVIATION  LEVEL*  (#  of  observers) 

DETECTED 


Lower  Upper 


Plus  X 

Panchromatic 

2042 

(Black  & 

White) 

44.7 

16.2 

38.1 

51.2 

26 

Infrared 
Aerographlc 
2424  (Black 
&  White) 

43.4 

18.4 

35.9 

a 

50.8 

26 

Aerochrome 

MS  2448 
(Color) 

55.4 

22.2 

46.1 

64.8 

24 

Aerochrome 
Infrared  2443 
(Color) 

58.6 

17.6 

D 

65.9 

25 

TABLE  4 


ONE  WAY  ANALYSIS-OF-VARIANCE  FOR  MEAN  PERCENTAGE  OF  TARGETS  DETECTED 

FROM  FOUR  TYPES  OF  AERIAL  FILM 


OURCE  OF  VARIATION  SQUARES  _ FREEDOM 


Types  of  Aerial  Film  I  4.428.8864 


33.812.6291 


38.412.5125 


MEAN  SQUARE 


1.476.2965 


348.5838 


F-RATIO 


*4.2351 


The  data  In  Table  5  Indicates  that  at  the  significance  level  <*  *  0.05, 
the  mean  detection  of  targets  on  the  Color  and  Color  Infrared  film,  was 
greater  when  compared  to  the  mean  detection  of  targets  In  the  Black 
and  White  Infrared  film  type.  The  mean  detection  of  targets  by  Image 
Interpreters  on  color  Infrarad  film  was  also  significantly  greater 
(i  0.05)  when  compared  with  the  mean  detection  of  targets  on  Panchro¬ 
matic  film.  An  even  more  significant  relationship  (a  ■  0.01)  was  f  und 
when  the  means  of  the  Color  and  Color  Infrared  films  were  combined  ar.d 
compared  with  the  combined  means  of  the  Panchromatic  and  Black  and  Wh4te 
Infrared.  Table  6  contains  the  means  for  the  number  of  targets  detected, 
as  well  as  the  95%  confidence  Intervals  and  sample  size.  Table  7  contains 
the  results  of  the  analysls-of- variance  performed  on  the  data  of  Tabic  6. 

6.0  DISCUSSION 

From  these  results,  and  assuming  the  Interpreters  to  be  of  equal 
experience  levels,  It  was  statistically  (a  *  0.05)  determined  that 
the  use  of  Aerochrome  Infrared  2443  film  resulted  in  a  greater  mean 
percentage  of  target  detections  than  that  of  either  Plus  X  Panchromatic 
2042  or  Infrared  Aerographlc  2424.  Aerochrome  MS  2448  film  also  allowed 
statistically  (a  -  0.05)  greater  mean  number  of  targets  detected  than  that 
of  Infrared  Aerographlc  2424.  The  mean  number  of  targets  detected  was 
combined  for  the  two  color  films  and  also  for  the  two  black  and  white  films. 
They  were  then  statistically  compared  against  each  other;  it  was  determined 
that  they  differed  at  the  0.01  level.  The  task  Involved  In  this  study 
was  basically  one  of  searching  an  unknown  area  of  film  for  the  detection 
of  military  targets,  some  of  which  were  embeded  In  trees.  The  resulting 
mean  percentage  for  target  detection  on  color  films  complements  some  of 
the  conclusions  of  a  US  Naval  Technical  Bulletin  5/  which  states  that 
color  photograph  provides  the  most  benefits  In  a  area  being  searched  for 
unknown  or  unlocated  targets.  The  bulletin  also  states  that  the  detect¬ 
ing  of  partially  hidden  targets  is  aided  by  the  use  of  color  Imagery  which 
provides  details  within  the  shadows.  Strandberg  stated  that  atmospheric 
haze  reduces  the  advantages  of  color  film  over  black  and  white  when  high 
obliques  or  horlzon-to-horlzon  panoramics  are  taken.  Therefore,  both  color 
and  black  and  white  Imagery  may  be  required.  It  is  Interesting  to  note 
that  64%  of  the  Image  interpreters  who  analyzed  the  Aerochrome  Infrared 
2443  film  stated  that,  with  the  exception  of  a  brief  session  in  school, 
they  have  not  had  further  experience  with  It.  Forty-two  percent  of  the 
Image  Interpreters  made  a  similar  statement  concerning  the  use  of  color 
film.  Accordingly,  It  may  be  that  given  additional  experience  with  these 
films,  the  mean  percentage  of  targets  detected  would  show  an  even  greater 
disparity  between  color  and  black  and  white  aerial  film  than  the  results 


211 


TABLE  6 


STATISTICAL  SUMMARY  FOR  PERCENTAGE  OF  TARGETS  DETECTED  FROM  THE  COMBINED 
MEANS  OF  COLOR  AND  BLACK  AND  WHITE  AERIAL  FILM 


Film  Type 

Mean  Percentage 
of  Target  Detected 

95%  Confidence  Interval 
Lower  Upper 

Standard 

Deviation 

i-mr 

mm 

Combined 

Color 

.  _  .  . 

57.1 

50.5 

63.6 

19. B 

49 

Combined 
Black  & 
White 

44.0 

38.5 

49.5 

17.2 

52 

Table  7,  below,  contains  the  results  of  the  analysls-of-varlance  performed 


In  the  data  of  Table  6. 


TABLE  7 

ONE  WAY  ANALYSIS-OF-VARIANCE  FOR  PERCENTAGE  OF  TARGETS  DETECTED  FROM  THE 
COMBINED  MEANS  OF  COLOR  ANO  BLACK  AND  WHITE  AERIAL  FILM 


Source 

Sums  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

. 

Between  Combined 
Color  and  Combined 
Black  and  White  Flltr 

I  1 

1 

4282.0657 

12.4833* 

Within  Types  of 
Aerial  Film 

up 

99 

343.0247 

Total 

38,241 .5125 

100 

Critical  Value:  F  ^gg  ■  8.29 
*  Indicates  significance  <  0.01  level 
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obtained  from  this  study.  Therefore,  the  quality  of  day  Imagery  Intelll- 
gency  should  Increase  with  the  Increased  use  of  Aerochrome  MS  2448  and 
Aerochrome  Infrared  2443  film  by  Image  Interpreters.  The  use  of  Infrared 
Aerochrome  2424  film  for  gathering  of  day  Imagery  Intelligence  Is  not 
justified  by  the  results  of  this  study.  Should  Aerochrome  Infrared  2443 
not  be  available,  or  night  photography  using  artificial  light  sources  Is 
desired,  Infrared  Aerograph 1c  2424  film  may  be  of  value. 

7.0  SUMMARY 

We  quantitatively  compared  the  target  detection  capabilities  of  101, 
US  Marine  trained,  operational  II 's.  They  analyzed  the  same  targets 
photographed  with  the  following  four  types  of  film: 

Aerochrome  MS  2448 
Aerochrome  Infrared  2443 
Infrared  Aerographlc  2424 
Plus  X  Panchromatic  2042 


We  found: 

a.  Aerochrome  Infrared  Images  resulted  In  significantly  greater 
(ot<  0.05)  mean  percentages  of  targets  detected  than  Plus  X  Panchromatic 
an<J  Infrared  aerographlc  Images. 

b.  The  mean  percentage  of  targets  detected  with  aerochrome 
Images  was  significantly  (  <.0.05)  greater  than  that  of  the  Infrared 
Aerographlc  film. 

c.  The  combined  mean  number  of  targets  detected  with  Image  types 
Aerochrome  MS  and  Aerochrome  Infrared  was  significantly  (  a“  0.01)  greater 
than  that  of  the  combined  mean  number  of  targets  detected  from  Plus  X 
Panchromatic  and  Infrared  Aerographlc  Imagery. 

From  the  above  results  It  Is  concluded  that  the  accuracy  of  day 
Imagery  Intelligence  will  Increase  with  usage  of  Aerochrome  MS  and  Aero¬ 
chrome  Infrared  Imagery  by  II' s. 


REFERENCES 


1.  Naval  Reconnaissance  and  Technical  Support  Center,  Image  Interoretatloi 
Handbook.  Vol  I,  TM  30-245,  December  1967. 

2.  Carl  Strandberg,  The  Language  of  Color,  the  Sensation  o.f.CQlgr. Jp&l 
of  Color  Aerial  Photography,  American  society  of  Photogrammetry,  rlrst 


Edition. 

3.  Eastman  Kodak  Comapny,  Kodak  Data  for  Aerial  Photogra_pM,  34th  Edition 
1971. 

4.  Dixon  and  Massey,  Introduction  to  Statistical  Analysis,  3rd  Edlt^ 
McGruw  Hill,  1969. 

5.  The  Naval  Reconnaissance  and  Technical  Support  Center,  An  Inyest 


;,  3rd  Edition, 


FT-HH 


/ll 


'j  UriitliitiAaei/lL'i  LnV».  AeiAk 


NEW  ALGORITHMS  FOR  NONLINEAR  LEAST  SQUARES 
AND  BAYESIAN  PARAMETER  ESTIMATION 

Warren  B,  St  wart  and  Jan  P.  Serena  an 
Chemical  Engineering  Department 
and  Mathematics  Reaearoh  Center 
University  of  Wisconsin 
Madison,  Wisconsin  53706 

New  algorithms  are  described  for  Bayesian  estimation  of  parameters  in 
nonlinear  models  of  multiple-response  systems.  Modal  and  interval  estimates 
are  provided  for  the  parameter  vector  6  of  the  predictor  model,  and  for  the 
variance-covariance  matrix  c  of  a  Normal  error  distribution.  Allowance  is 
made  for  gaps  (missing  values  of  responses) ,  such  as  commonly  occur  in 
practice.  Two  chemical  examples  are  analyzed. 

INTRODUCTION 

Realiatia  models  of  multivariate  phenomena  often  relate  several  predicted 
responses  to  a  common  set  of  parameters.  Multiresponse  experiments  are  re¬ 
quired  to  establish  such  models,  but  frequently  yield  irregular  data  which 
are  difficult  to  analyze  by  classical  methods. 

Bayes'  theorem  ia  a  good  starting  point  for  parameter  estimation  in  these 
situations.  The  multivariate  error  distribution  can  be  estimated  concurrently, 
whereas  it  has  to  be  prescribed  when  least-squares  methods  are  used.  ThuB, 
the  Bayesian  approach  allows  more  objective  parameter  estimates,  if  sufficient 
data  are  provided.  An  excellent  general  account  of  this  approach  is  given  by 
Box  and  Tiao  (1973) . 

Bayesian  inference  deals  with  a  data  array  {y^}  =  y,  a  model  for  E(y) 
with  parameter  vector  6  ,  and  an  error  distribution  model.  If  a  Normal  error 
model  is  used,  with  variance-covariance  matrix  a  ,  the  unknown  elements  of 
o  will  appear  as  additional  parameters.  The  full  set  of  parameters  can  be 
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estimated  optimally  by  maximising  the  posterior  density  p(6,a|y)i  confidence 

*  w 

regions  can  also  be  calculated  from  this  function. 

In  aertain  cases,  the  posterior  density  can  be  integrated  analytically 
to  obtain  the  marginal  density  p(§|y).  Box  and  Draper  (1965)  accomplished 
this  for  multivariate  Normal  error  distributions  and  rectangular  data  structures 
(Table  la).  For  bloek-reatangular  structures  (Table  lb),  p(0|^)  is  the  prod¬ 
uct  of  the  Box-Draper  densities  for  the  individual  rectangles.  More  compli¬ 
cated  data  structures  often  occur,  however,  such  as  that  in  Table  lo,  for 
which  p(e|^)  cannot  be  expressed  in  closed  form.  Therefore,  in  this  paper 
wo  use  the  full  posterior  density  p(6,c|y),  which  has  a  dosed  form  for  any 
finite  data  structure. 

Inspection  of  the  parameter  estimates  and  residuals  often  suggests 
alternatives  to  the  postulated  model.  Therefore,  parameter  estimation  should 
not  be  viewed  as  an  end  in  itself,  but  should  be  followed  by  critical  examina¬ 
tion  of  the  model  and  investigation  of  any  promising  alternatives.  Interesting 
predictions  or  unresolved  differences  between  models  will  naturally  lead  to 
further  experiments . 
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PROBLEM  FORMULATION 


Consider  a  set  of  independent  experiments,  u  -  l,...,n,  in  whiah  a 
table  {yuiJ  of  observed  responses  have  been  obtained  at  known  settings 
{x  }  of  the  independent  variables.  There  aro  m  linearly  independent 
kinds  of  observations  >  thus  the  index  i  ranges  from  1  to  m  ,  but  in 
each  experiment  some  values  may  be  missing  as  in  Tables  lb  and  la. 

The  observations  in  the  uth  experiment  are  regarded  as  a  sample  from 
a  population  of  the  form 

»«i  ■  +  ‘ui  •  111 

The  functions  f^(xu,e)  are  models  for  the  expected  responses  HJ (yuj_  I §)  • 

The  residualB  in  the  uth  experiment  are  treated  as  a  random  sample 

from  an  m-variate  Normal  distribution t  this  gives  the  probability  density 


(Wilks,  1962) 

p(eu|o)  -  <2ir)  u  |gur1/2  sxp(~i  Eu>  -  (2> 

Here  e  is  the  column  vector  of  error  variables  c  with  dummy 

u  UJi  urn 

zeroes  inserted  where  observations  are  missing.  Correspondingly,  is 

obtained  from  the  full  variance-covariance  matrix,  a  ■  {cr^}  •  sub- 
atituting  dummy  elements  6^  whenever  observation  y^  or  yy^  is 
missing.  Here  6^  is  unity  when  i« j ,  and  zero  otherwise. 

The  joint  error  density  model  for  the  set  of  n  experiments  follows 


direatly  from  Equation  (2) i 
n  -m  /2 


p(e|e,a)  -  II  <2 7T )  u  | o  l"1'2  exp(-i  e  o”  e  ).  (3) 

J  ~  ' -u '  ~u  ~u  ~u 

insertion  of  Equation  (1)  gives  the  corresponding  density  in  observation 


space ; 


p(X|e,a)  ■  t  It  (2ir)  u  | | ] 
u-1 


n  m  m 


expM  l  l_  J_  [yul  -  fui<g)Hyu1  - 


u-1  i-1  j-1 


uj  *uj 


■  <•  U.i,  slKi. 


Hare  the  functions  fui(6)  stand  for  f^(xu#§)  evaluated  at  the  known 
settings  xy  of  the  independent  variables.  The  ere  the  elements  of 

the  preaision  matrices  a”3-.  The  right-hand  term  may  also  be  regarded, 
by  Bayes'  theorem,  as  the  likelihood  function  for  9  and  a  when  eval¬ 
uated  with  given  observations  y  . 

The  usual  factorisation  oil  v.ho  prior  density  p(0,g)  ia  assumed, 
p(9,c)  -  p(9)  p(cr)  (5) 

M  a*  «w 

and  a  locally  uniform  density  p(9)  is  assumed  in  the  region  of  appreci- , 
able  likelihood.  The  latter  assumption  requires  some  care  in  the  parame- 
trization  of  the  model.  The  prior  density  of  o  is  taken  from  Box  and 


Draper  (1965) i 


•<m+l)/2 


Bayes*  theorem  then  gives  the  posterior  donsity 

p<s# “  p(§*2.) 

-  c|or<m+1>~/2 1 5  uurV2]  <7) 

U“1  u 

•  X  jJ1  °l3tyui  “  fui(§)]  tyuj  -  fuj(->]} 

in  whioh  c  is  a  proportionality  constant.  All  that  the  data  reveal  about 
the  parameters  6  and  o  is  contained  in  this  density  function. 

Point  estimates  of  0  and  a  are  obtainable  by  maximizing  the  posterior 
density  just  described,  or  by  minimizing  the  function 

=  S(9,0)  -  -2  In  p(9,ojy)  +  2  In  c 

m  •*  m,  _  m  a* 

n 

-  (m+l)  In  |a|  +  l  In  |a  |  (8) 

u-1 

n  m  m  ,  . 


‘iui.  I  "«  -  V!>' 


over  the  permitted  region  of  6  and  a  .  Here  ij;  is  a  column  array  of 

the  model  parameters  e.,  . ..,6  and  the  independent  elements  of  g  . 

*  P 


*  nrv. t 


The  latter  are  taken  from  the  lower  triangle  of  o  in  row  order ,  i„e. 

•W 

ip  “  Ojj  with  i  >  j  and  k  -  j  +  i(i-l)/2.  Thus,  the  total  number 
p+x  ij  ■“ 

of  parameters  is  q  ■  p  +  m(m+l)/2. 

If  the  matrix  a  were  believed  to  be  known,  i.e,,  if  a  sharply  focussed 
prior  density  p(g)  were  assumed,  then  S  (ip)  would  reduaa  to  S(6)  and  we 
would  have  a  least-squares  estimation  problem  with  just  p  parameters.  In 
practice,  one  seldom  knows  a  accurately;  hence,  the  full  Bayesian  solution 
is  recommended. 


PARAMETER  ESTIMATION  ALGORITHMS 


Several  algorithms  are  described  here  for  obtaining  summary  informa¬ 
tion  from  Equation  (8) .  These  algorithms  are  part  of  a  Fortran  IV  package 
available  from  the  authors. 

1.  Counting  Algorithm 

Before  analysing  S  we  count  Equations  (1)  to  see  which  parameters 
can  plausibly  be  estimated  from  the  data.  We  first  try  to  match  each 
parameter  in  \J >  with  an  observation  pair  ^uv;'yuj^  o:f  *  replicate 

experiment  U. e_. ,  an  experiment  which  has  the  same  oxpeated  response  values 
as  a  prior  experiment  in  the  data  set) .  If  this  process  cannot  be  completed 
for  a  given  k  ,  we  then  try  to  match  each  remaining  error  parameter  , 

and  each  model  parameter  6£  in  the  function  pairs  fuj(6)]»  with 

a  non-replioate  observation  pair  anV  remaining  model 

parameters  @r  are  matched  with  remaining  non-replicate  observations.  If 
the  matching  can  be  completed  for  all  el  aments  of  ,  we  proceed  with  the 
estimation.  Otherwise,  the  full  sot  of  parameters  aannot  be  estimated  from 
the  data. 

The  counting  algorithm  is  a  logical  Gaussian  elimination.  This  test 
is  a  useful  diagnostic,  but  is  not  infallible,  since  the  actual  rank  of  the 
estimation  equations  depends  on  the  numerical  values  of  x,  y,  and  iji . 


2.  Minimization  Algorithm 

A  modified  Newton  method  is  used  to  find  a  minimum  of  S(i|i).  Let  iji0 
be  the  value  of  tjj  at  the  start  of  an  iteration.  A  correction  vector 


(i|i.  -  i  )  is  computed  by  minimizing  the  local  quadratic  expansion  (see 


Appendix  A  for  derivative  expressions) 


■<!>  “  3%]  +ff 


10  - 


{*  -  J0)  +  J  Hj  -  *0)T  ^1 


.<? "  !o) 
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ovar  a  user-specified  rectangular  region  around  .  The  region  is  choaen 
■mall  enough  to  eneure  that  S(iji)  ia  a  good  approximation  to  the  function 

M 

S(iJj)  of  Equation  (9).  a  search  ia  then  made  for  a  minimum  of  S  in  the 
interval  of  positive  definite  a  on  the  line  from  ip  through  tj*.  ;  this 

e  N  ^  N* 

gives  the  starting  point  for  the  next  iteration.  The  calculation  continues 
until  two  successive  line-minima  agree  within  confidence  intorvals  calculated 
from  Equation  (14)  for  each  parameter. 


3.  Response-independence  Test 

Box  and  co-workers  (1973)  have  pointed  out  the  need  to  test  the  responses 
for  linear  independence.  Preferably,  one  should  perform  this  test  on  the 
residuals  [y^.,  -  fyi(6)],  which  might  become  linearly  dependent  in  oertain 
regions  of  6  .  Xn  the  present  procedure,  such  linear  dependence  is  readily 
deteated  during  the  inversion  of  a  at  the  start  of  each  iteration.  The 

m* 

calculation  can  continue  if  all  pivot  elements  (Stewart,  1973)  found  in  this 
inversion  are  greater  than  a  specified  fraction,  say  0.1,  of  the  corresponding 
elements  . 


4.  Confidence  Regions 

Equation  (8)  gives  the  simple  form 

p(ijj|y)  «•  exp [-i  S(tJj)]  (10) 

for  the  posterior  density  function,  or  "confidence  density",  use  of  Equation 
(9)  gives  the  approximation 

p(ljj|y)  «  exp(-4(ij<  -  $)T  -  $)  ]  ,  (11) 

^  *>  n  at  w  at 

A 

valid  in  the  neighborhood  of  the  minimum  point  <|i  .  Here  A  is  the  qxq 
matrix  (positive  definite  since  S  is  at  a  minimum)  with  elements 


A  i  32S 
km  "  3  3<ik94i 


m 


(12) 
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computed  as  described  in  the  Appendix.  Thus,  near  the  optimum,  the  param¬ 
eters  ere  Normally  distributed  with  variance-covariance  matrix  A-2,  .  If 
Equation  (11)  is  used  as  an  approximation  for  all  values  of  ip  ,  then  the 
confidence  intervals  for  Normal  distributions  aan  be  applied.  For  example, 
the  ellipsoidal  region 

(tp  —  $)T  it(4»  -  $)  <  x2  (q,a)  (13) 

w  «■  At  nt 

roughly  approximates  the  100(1  -  a)  percent  highest-postsrior-density  region, 
or  joint  confidence  region,  for  tj*  based  on  the  given  data.  The  intervals 
-  y/  )  <  erfc“1(a>  (14) 

roughly  approximate  the  100(1  -  a)  percent  confidence  intervals  for  the  indi¬ 
vidual  parameters.  For  symmetric  95  percent  confidence  intervals  (a  ■  0.05), 
erf  o’*1  (a)  has  the  value  1,96. 

Equation  (14)  is  more  reliable  than  (13),  since  the  integration  used  to 
obtain  it  is  lass  affected  by  the  tails  of  the  posterior  density  function. 

Mora  aaourata  intervals  can  be  obtained,  but  with  greater  effort,  by  numerical 
integration  of  Equation  (7)  or  (10), 


222 


RESULTS  FOR  RECTANGULAR  DATA  STRUCTURES 

If  every  experiment  gives  a  full  eet  of  observations  yul» .  ♦  »y„m  * 


'urn 


then  Equation  (7)  takes  the  form 


m  m 


p(6,a|y)  «  |a|-(m+n+l)/2  exp[_|  £  J  v.. (8)J 

“  “  "  i-1  j-1  13 


(15) 


in  which 
v 


ii‘21  "  Jj  "ui  ‘  £ui(!,llyuj  ‘  'uj'!11  •  ll6» 

Integration  of  Equation  (IS)  over  the  region  of  positive  definite  o  gives 

<v 

the  marginal  density  function 

P(9|y)  *  |v(0)|"n/2  (17) 

•»  mt  ** 

as  shown  by  Box  and  Draper  (1965) .  We  wish  to  compare  the  estimates  based 
on  this  function  with  those  obtained  from  the  full  density  function  of 
Equation  (15). 

Setting  p(0|y)  stationary  with  respect  to  its  parameters  gives 


a  In  |v<0) |  i  3v 

- yr  **—  s  l  l  V  3  -  0 

30k  i  j  &6k 


1  I  •  •  •  fP 


(18) 


when  use  is  made  of  the  Laplace  expansion  of  jv  +  dv|.  Here  the  v^  are 

w  **• 

the  elements  of  the  matrix  v”*’  . 

Setting  p(6,c|y)  stationary  with  respect  to  its  parameters  gives,  after 
use  of  Equation  (15) , 


3  In  p(0,a|y)  ,,  3v  (0) 

-2  — ae^- 2  \  \  ° j  "  0  k  - 1 . . 

-  a  In  p(6,a|y)  a  „ 

-2  -  a.  ff-a.  =  (m+n+1)  (In  |o|)+  \  l  c^v.  (0) 

aor>  acrB  -  acr“  i  j  13  - 


(19) 


<2  -  6  )(-(m+r.+l)  a  +  v  (0)]  ■  0 

rs  rs  rs  ~ 


(20) 


T  m  1  f  e  e  •  #IU  8  * 
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Equation  (20)  gives,  at  the  stationary  point, 

V  (8) 

*  ra  » 


ra  m+n+1 


Hence, 

orB  »  (m+n+1)  vrs(8)  .  (22) 

Insertion  of  Equation  (22)  into  (19)  gives  Equation  (18)  at  the  stationary 
point  of  p(9,o|y).  Hence,  for  rectangular  data  structures,  the  same  values 
of  8  and  o  are  obtained  whether  one  maximizes  p(9,o|y)  or  p(8|y). 

•a  •*  •*  •*  **  **  •* 

Of  courae,  the  marginal  confidence  regions  for  0  can  be  estimated  more 
directly  in  the  latter  oase.  The  normal  equations  based  on  p(o|y),  given 

•m  a* 

by  Stewart  and  Sorensen  (1976) ,  are  convenient  for  this  purpose. 

The  covariance  estimates  in  Equation  (21)  are  maximum-density  values, 
and  thus  differ  from  the  expectation  values  Elo  |y)  unless  n-m-p  is 
very  large.  If  expectation  estimates  of  the  a  are  desired,  one  oan  com- 
pute  them  as  the  corresponding  moments  of  the  normalized  posterior  density 
p(9,c|y) . 

EXAMPLE  1.  Kinetics  of  a  Three-Component  System 

Consider  the  chemical  conversion  of  initially  pure  species  1  to  species 
2  and  3  in  a  batch  isothermal  reactor.  Simulated  data  for  the  system  are 
given  in  Table  1,  reproduced  from  Box  and  Draper  (1965),*  here  yy^  is  the 
yield  of  speciee  i  in  experiment  u  ,  The  system  is  modelled  by  the  differ- 

I 

ential  equations 


“sr "  -ki  fi 


kl  fl  "  k2  £2 


•df  “  k2  f2 

which  have  the  solution 


aitiiiLKinb  .SisaMdinat&uiulfiLUVjuiuildiuirv 


jj'jlL  l-  i  uai-iu  ' i  ■  mi  V  ip"  i  i"*i  i'-'i  Hi  1 1 1  'i*i  i’lii)  ii  j'il'iiiidtilijtfjiii  jfci. ! ' 


^  ■  expt-^  t) 


f2  =  [expt-k^  t)  -  exp(-k2  t))kj/(k2  -  k^ 
fJ  "  1  *  fl  -  f2 

under  the  indicated  initial  conditions.  As  noted  by  Box  and  Draper,  it  is 
natural  to  regard  the  parameters  0^.  ■  In  k^  as  uniformly  distributed  a  priori . 

There  are  three  responses  yyi  per  experiment.  Only  two  would  be  linearly 
independent  if  the  yields  were  mass-balanced  (i.e.,  if  the  yieldB  in  each  row 
added  up  to  unity).  The  data  in  Table  2  are  clearly  not  mas ^-balanced,  so  we  use 
all  three  columns  of  responses. 

The  repli'  ates  in  Table  2  allow  preliminary  estimation  of  the  parameters 

0.  , ,  by  the  relation 
■*  nR 

■lj  ’  iS;  (lri  '  »k>  (yrj  ‘  'rj1  ' 

Here  y^  and  y^  are  the  observations  of  response  i  in  the  first  and 
second  tests  of  replicate  pair  r  ,  and  n  is  the  number  of  such  pairs. 

This  procedure  gives 

‘  0,00102  -0.00128  0.00025 

{si j  }  -  • -0.00128  0.00351  0.00024 

0.00025  0.00024  0.00101 

as  a  preliminary  expectation  estimate  of  o  .  This  is  a  well-conditioned 
matrix,  so  our  choice  m  ■  3  was  correct. 

The  parameter  vector  ^  for  the  present  example  consists  of  0^,  02, 
and  the  six  elements  on  and  below  the  diagonal  of  a  .  To  test  the  conver¬ 
gence  of  the  estimation  from  a  poor  initial  guess ,  the  calculation  was  started 
from  the  initial  value  shovrn  in  Table  3.  Convergence  was  obtained  in  eight 
iterations,  to  the  point  estimates  and  35  percent  confidence  intervals  given 
there . 
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A  second  calculation  was  made  with  the  same  initial  values,  but  with 
second-order  6-derivatives  neglected.  Convergence  was  obtained  to  the  same 
joint  estimates  in  nine  iterations.  The  confidence  intervals  differed 
slightly,  as  shown  in  Table  3, 

A  third  calculation  was  made  by  minimizing  the  determinant  jv(6)|.  Box 

v.  •«  M 

and  Draper  (1965)  did  this  by  a  search  procedure;  we  used  the  modified  Newton 
algorithm  of  Stewart  and  Sorensen  (1976),  but  neglected  the  second-order  6- 
derivatives  of  the  functions  (6)  .  Convergence  was  obtained  in  seven 
iterations,  to  the  same  point  estimates  6^  ,  The  point  estimates  for  the 

,  computed  from  Equation  (21),  alBo  agreed  exactly  with  the  two  preceding 
solutions.  The  one-parameter  confidence  intervals  (computed  in  this  case  only 
for  6^  aid  9  )  are  wider  than  before,  and  are  considered  more  accurate 
since  in  this  case  the  have  been  integrated  out  exaatly  (Box  and ‘Draper, 

1965) . 

EXAMPLE  2.  Kinetics  of  a  Five-Component  System 

Fuguitt  and  Hawkins  (1945,  1947)  did  extensive  experiments  on  the  liquid- 
phase  thermal  reactions  of  a-pinsne  and  its  decomposition  products.  The 
following  products,  in  order  of  boiling  point,  were  identified. 


A. 

a-Pinene 

C10H16 

a. 

a  -  and  0-Pyronene 

C10H16 

c. 

Dipentene 

C10H16 

D. 

allo-Ocimene 

C10»16 

E. 

Dimer 

C20H32 

The  reaction  conditions  and  yields  are  reported  in  Table  4. 

We  have  normalized  the  yields  to  obtain  exact  mass  balances;  this  makes 
the  yields  linearly  dependent,  and  accordingly  we  have  omitted  species  D  . 
The  remaining  species  are  grouped  aB  cumulative  distillation  fractions: 
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i 

? 

A,  A+B,  a+B+C,  and  E.  Eaah  of  these  responses  represents  essentially  the  total 
mass  fraction  distilling  above  or  below  a  particular  temperature.  The  yields 

i 

of  B  originally  reported  in  testa  1-15  have  been  deleted,  since  they  were 
interpolated  values  rather  than  observations  (Fuguitt  and  Hawkins,  1947 j  Box 
and  co-workers,  1973). 

There  are  numerous  gaps  in  the  data.  &-Pinene  (a)  was  reported  in  experi¬ 
ments  1-16,  but  was  considered  negligible  in  the  remaining  experiments.  Pyronenes 
(B)  were  reported  only  in  experiments  16-31j  they  proved  difficult  to  isolate 
except  at  small  concentrations  of  a-pinene.  Only  the  dimer  fraction  (E)  was 
reported  in  the  experiments  with  allo-ooimena  <D)  or  dimer  (E)  as  foed.  The 
simplified  reaction  scheme  proposed  by  Fuguitt  and  Hawkins  (1947)  implies  that 
a-pinene  (A)  and  dipentene  (C)  would  not  be  formed  in  the  latter  experiments, 
but  that  the  other  three  species  would  be  present. 

The  first  eight  experiments  were  used  for  parameter  estimation  according 
to  Equation  (17)  with  m  -  3  by  Box  and  co-workers  (1973) ,  and  by  the  present 

i 

[  authors  (1976) .  The  full  41  experiments  could  not  be  so  analysed  beaause  of 

F 

their  irregular  structure;  therefore,  only  rough  estimates  were  obtainable  for 

( 

several  of  the  reaction  parameters.  With  Equation  (8),  on  the  other  hand,  all 
41  experiments  can  be  analyzed. 

We  postulate  the  following  reaction  scheme, 


with  the  following  differential  equations  for  the  concentrations « 


<♦,  2 
s£  -  -(iti  *  v  ♦»  -  2ts  ♦; 


8 


d«j». 

"dt 

d*C 

St 

d(j> 

"dt 

d<6_ 


"  "k-3  *B  +  *1* 


3  ’D 


kl  *A 


D  “  k2  *A  +  k-3  "  k3  *D  "  2k4  +  2k~ 


■3  TB 
,2 


4 


-sr*  “  kB  +  k .  -  k  -  4>_ 

dt  5  A  4  tD  -4  TE 

Here  we  have  assumed  equal  densities  for  the  reaction  mixture  and  all  species. 
The  i _  are  molar  concentrations  relative  to  the  molar  density  of  pure  liquid 
orpineue  at  the  reaction  temperature .  The  resulting  initial  values  for  the 

pure  reaatants  aroi  1.0  for  a-pinana,  1.0  for  allo-ooimane,  and  0.5  for 
dimer.  The  rate  coefficients  are  represented  as  Arrhenius  functions. 

In  (k4)  -  61  -  (l/T  -  i/tb)  ei+5  i  -  i,...,5 
In  (k3/k_3)  -  -ell/TB  +  (l/T  -  1/TB>  e13 
In  (k4/k_4)  -  -6la/TB  +  (l/T  -  1/Tb)  014 
with  k^  values  in  min”1,  T  in  Kelvins,  and  a  base  temperature  TB  of 


47B.5  K. 


The  data  and  parameters  were  paired  to  check  the  feasibility  of  the 
estimation.  ThiB  indicated  a  sufficient  amount  of  data  for  estimation  of  all 
parameters  except  .  However,  the  replicate  comparisons  (u  ■  18-19,20-21, 

22-23,24-25)  involving  yy2  all  give  duplication  of  yu3>  furthermore  each  of 
these  comparisons  gives  a  duplication  of  either  yy2  or  y^.  With  these 
results,  we  find  that  neither  o32  nor  a 42  can  be  estimated)  indeed,  an 
attempt  to  estimate  them  was  terminated  by  the  linear  independence  test 
described  above.  Thereafter,  c2^,  o32,  and  o^2  were  all  fixed  at  zero,  and 
the  remaining  parameters  were  estimated  by  minimization  of  S  . 


I 
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Initial  valuei  of  the  6-parameters  were  chosen  from  the  results  of 
Fuguitt  and  Hawkins  (1945,  1947) ,  Box  and  co-workers  (1973) ,  and  the  present 
authors  (1976) .  initial  variance  estimates  were  calculated  from  repli¬ 

cate  data  available  in  Table  4,  and  zeros  were  inserted  initially  as  covariances. 

The  model  was  integrated,  for  each  experiment ,  by  the  method  of  Guertin 
et  al  (1977)  with  6  mesh  points.  The  coefficients  in  Equation  (9)  were  com¬ 
puted  as  described  in  the  Appendix,  with  first-order  sensitivities 
computed  by  the  method  of  Stewart  and  Sorensen  (1976) . 

A  first  minimization,  with  roaction  5  omitted,  converged  within  20  itera¬ 
tions.  This  gave  £  -  41.06  with  parameter  estimates  as  shown  in  Table  5. 

The  confidence  intervals  show  the  6*s  to  be  estimated  quite  precisely.  The 
are  estimated  less  precisely,  as  anticipated  from  the  limited  number  of 
data  on  several  combinations  of  responses.  The  deviations  of  the  data  from 
the  fitted  model  are  shown  in  Table  6. 

A  second  minimization  of  S  was  done  with  the  full  5-reaction  model. 

This  calculation  converged  to  a  very  flat  minimum  at  S  ■  34.09,  with  param¬ 
eter  estimates  as  shown  in  Tabic  5.  The  deviations  of  the  data  from  this 
fitted  model  arc  also  shown  in  Table  6. 

The  L-reaction  model  is  better  able  to  describe  the  polymer  yields  from 
o.-pinene  at  short  times,  as  can  be  seen  in  Table  6.  We  can  also  test  the 
significance  of  the  added  parameter  0,.  by  use  of  the  confidence  intervals. 
Table  5  gives  6^  -  -11.945  ±  0.698,  based  on  Equation  (14) >  this  implies  the 
limits  (.1  ±  0.698)  exp(-11.945)  for  k,.  with  the  alternate  prior  p(kg)  -  c. 
Hence,  the  95%  confidence  interval  for  kg  dees  not  include  zero. 

On  the  other  hand,  Equations  (9)  and  (13)  give  the  following  approximate 
expression  for  the  95%  confidence  region  of  the  20  fittad  parameters  of  the 
5-reaction  model i 
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S<£)  -  34.09  <  Xjq (0.05)  -  31.41. 

All  ip  values  »uch  that  s (0)  <  65.50  11a  within  thla  aatimatad  95%  joint 

•*  •* 

confidence  ragion.  By  this  criterion,  tha  modal  with  k£  0  is  aooaptabla. 
However,  as  indioatad  earlier,  Equation  (14)  ia  mors  raliabla  than  (13).  From 
this,  and  a  study  of  tha  rasiduala,  wa  oonoluda  that  tha  5-raaction  modal  is 
to  ba  prafarrad. 
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APPENDIX t  DERIVATIVES  OF  S 


Tha  matrices  o  ara  real  and  symmetric)  furthermore,  S  is  defined  only 

M  » 

when  these  matrices  are  positive  definite ,  The  following  .derivative  relations 
then  holdi 


3  in  |oJ 


(2  - 


j  <  i 


—  -  -i  (2  -  <5.  „ )  CcJ?:k  o*j  +  0 .U  okj]  A  <  k. 


kA'  lwu  Uu  T  wu  Uu 


The  relations  for  second  derivatives  follow  by  combination  of  (Al)  and  (A2) t 

2  |  | 

■  -*(J  -  v (s  -  *«>  +  ° H)]  J  i  *- 1  i * 


*W,<’uk* 


,,  is  tk  it  sk.  Aj  ik  ,  As  tj  J  At  sj. 

t(o  a  +cr  a  )  a  J  +  a  (a  a  J  +  a  oJ) 
uu  uuu  uuu  uu 

is  tA  it  sA.  kj  iA  ,  ks  tj  kt  sj 
+  (ou  °u  +  au  au)  °u  +  °u  (au  au  +  au  °u  )] 

A  <_  k  ,  t  <  b  i  (A4) 

As  indicated  earlier,  if  response  h  is  absent  from  experiment  u  ,  the 
elements  and  are  replaced  by  tha  constant  dummy  values  6^  . 

Note  alao  that  the  symmetry  of  oy  has  been  used  to  express  theae  deriva¬ 
tives  in  terms  of  elements  on  and  below  the  diagonal. 

The  derivatives  required  for  Equation  (9)  are  obtained  as  follows > 


"ll  i  <2  “  6ijJ  V*  ae^  <eui  euj) 


V  V  A  I  u 

r  u  i  j<i  J 

*»rsl —U  l 


"r3ev 


u  i  j  <i 


ijJ  °u  aV  33  {eui  Euj} 
r  v 


Parameter 

Table 

3.  parameter  Values  for  Example 

1 

Solution  3 
Eq*.  (18,21)* 

Initial 

Value 

Solution  1 

Eqs.  (8.14)* 

Solution  2 
Eqs.  (8,14)* 

61 

-2.3026 

-1. 572310.0567 

-1.572310.0558 

-1.572310.0800 

62 

0. 

-0.702310.1374 

-0.702310.1346 

-0.702310.1931 

°11 

0.01 

(0.7610.52)  10"3 

(0.7610.53) 

1C"3 

0.76  10”3 

°21 

0. 

-(0.5010.63)  10"3 

-(0.5010.63) 

io"3 

-0.50  10”3 

022 

0.01 

(1.8611.28)  10-3 

(1.8611.29) 

io’3 

1.86  IO-3 

°31 

0. 

(0.3210.41)  10“3 

(0.3210.41) 

io"3 

0,32  10“3 

°32 

0. 

(0.4010.62)  10"3 

(0.4010.62) 

io'3 

0.40  10“3 

°33 

0.01 

(0.7710,54)  10"3 

(0.7710.54) 

io“3 

0.77  10“3 

* 

All  Intervals  are  SSI  highest  posterior  density  regions,  in  Solution  3,  the 
intervals  are  computed  from  the  normal  equations  with  "residual  mean  square" 

a 

J v (8) | / (n— 2)  and  n~2  ■  10  residual  degrees  of  freedom,  in  Solution  1,  the 
second-derivative  terms  of  Equation  IA10)  are  included. 
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Table  4.  Data  for  Example  2,  from  Fuquitt  and  Hawkins  (1945,1947) 

Nomaliged  yields,  weight  peroent 


Expt, 

u 

Feed 

T,  C. 

tu,min 

w 

u 

yul 

(A) 

yu2 

<A+B) 

*u3 

(A+B+C) 

yu4 

(E) 

1 

A 

189.5 

1230. 

1 

88.3 

*** 

96.2 

2.2 

2* 

A 

189.5 

1230. 

1 

88.2 

*** 

95.7 

1.3 

3 

A 

189.5 

3060. 

2 

76.4 

*** 

92.7 

2.8 

4 

A 

189.5 

4920, 

2 

64.8 

88.9 

5.8 

5 

A 

189.5 

7800. 

2 

50.3 

*** 

84.7 

9.3 

6 

A 

189.5 

10680. 

2 

37.5 

*** 

82.0 

12.0 

7 

A 

189.5 

15030. 

2 

25.9 

*** 

77.1 

17.0 

a , 

A 

189.5 

22620. 

2 

14.0 

73.9 

21.0 

9 

A 

204.5 

440. 

2 

86.6 

*** 

95.3 

.6 

10 

A 

204.5 

825. 

2 

75.0 

*** 

91.5 

1.6 

11 

A 

204.5 

1200. 

2 

66.0 

*** 

88.8 

3.4 

12 

A 

204.5 

1500. 

2 

59.4 

*** 

86.4 

5.1 

13 

A 

204.5 

2040. 

2 

48.9 

*** 

83.0 

8.3 

14 

A 

204.5 

3060. 

2 

32.8 

*** 

77.8 

13.8 

15 

A 

204.5 

6060. 

2 

11.5 

*** 

70.4 

22.5 

16 

A 

189.5 

36420. 

2 

4.5 

7.4 

70.5 

25.7 

17 

A 

204.5 

16020. 

2 

x 

3.1 

66.2 

28.6 

18 

A 

225.0 

3000, 

1 

a* 

3.0 

66.0 

28.0 

19* 

A 

225.0 

3000. 

1 

4.0 

66.0 

28.0 

20 

A 

245.0 

630. 

1 

- 

4.0 

65.0 

27.0 

21* 

A 

245.0 

630. 

1 

we 

5.0 

65.0 

27.0 

22 

A 

265.0 

120. 

1 

- 

7.0 

65.0 

23.0 

23* 

A 

265.0 

120. 

1 

- 

7.0 

65.0 

24.0 

24 

A 

285.0 

30. 

1 

- 

11.0 

66.0 

19.0 

25* 

A 

285.0 

30. 

1 

- 

9.0 

66.0 

19.0 

26 

D 

189.5 

1020. 

1 

- 

- 

- 

80.0 

27 

D 

189.5 

3990. 

1 

- 

- 

- 

87.3 

28* 

D 

189.5 

3990. 

i 

- 

- 

- 

87.3 

29 

D 

189.5 

6780. 

1 

- 

- 

- 

87.5 

30 

D 

189.5 

8220. 

1 

- 

- 

- 

86.5 

31 

D 

189.5 

13260, 

1 

- 

- 

- 

88.5 

32 

D 

189.5 

14760. 

1 

- 

a* 

- 

89.6 

33 

D 

204.5 

3480. 

1 

- 

- 

- 

87,5 

34 

D 

204.5 

5700. 

1 

- 

- 

- 

86.8 

35 

E 

189.5 

8880. 

1 

- 

w* 

- 

91.9 

36* 

E 

189.5 

8880. 

1 

- 

- 

- 

92.0 

37 

E 

189.5 

14340. 

1 

m 

- 

- 

89.8 

38 

E 

189.5 

23400. 

1 

- 

- 

- 

89.7 

39* 

E 

189.5 

23400. 

1 

- 

- 

- 

88.5 

40 

E 

204.5 

5700. 

1 

- 

- 

- 

88.4 

41 

E 

204.5 

8100. 

1 

- 

- 

- 

87.9 

it  ~  " 

Replicate  of  the  preceding  test. 


wu  is  the  number  of  independent  teats  combined  to  obtain  each  observation  y 

•kirk 

Originally  reported  but  not  observed;  see  text. 

-  No  value  reported. 


Table  5.  Pttrgmgtgra  for  g-Pinena  Converglon 

* 


Parameter 


Estimates  for 
4-Reaction  Model 


Estimates  for 
5-Reaction  Model 


91 

-8.331 

± 

.024 

-8.333 

4 

.025 

02 

-8.898 

± 

.029 

-8.961 

4 

.054 

93 

-8.242 

± 

.341 

-8.196 

4 

.325 

64 

-5.389 

4 

.081 

-5.438 

4 

.087 

fl5 

-11,945 

4 

.698 

06 

19814. 

4 

428. 

19785. 

4 

457. 

97 

20828. 

± 

474. 

20890. 

± 

536. 

98 

17336. 

± 

4079 , 

17212. 

i 

4203. 

°9 

10321. 

± 

915. 

10322. 

4 

918. 

eio 

19957. 

** 

•u 

269. 

4 

83. 

279. 

± 

83. 

012 

-1976. 

± 

64. 

-1985. 

± 

63. 

013 

-336. 

± 

950. 

-259. 

± 

958. 

9 14 

-3873. 

± 

1624. 

-3781. 

4 

1555. 

ffll 

.696 

± 

.419 

.784 

± 

.492 

•u 

.000 

** 

.000 

** 

a22 

.391 

± 

.359 

.376 

4 

.348 

°31 

.358 

i 

.412 

.426 

4 

.456 

°32 

.000 

** 

.000 

** 

a33 

.706 

± 

.426 

.732 

4 

.444 

°41 

-.248 

4 

.344 

-.294 

± 

.354 

•«, 

.000 

** 

.000 

** 

°43 

-.504 

4 

.317 

-.493 

t 

.314 

°44 

.744 

4 

.304 

.654 

± 

.282 

95%  highest  posterior  density  intervals  calculated  from  Equation  (14) , 
** 

Posterior  estimates  were  not  obtained  for  these  parameters. 


Expt. , 
u 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 


Table  6.  Final  Residuals  eui(9)  for  Example  2. 


4-Reaction  Model  5- React ion  Model 


cul 

(A) 

6uS  . 
(At*: 

Cu3 

(A+B+C) 

eu4 

(E) 

Eul 

(A) 

eu2 

(A+B) 

eu3 

(A+B+C) 

-1.32 

-.37 

2.00 

-1.22 

_ 

-.26 

-1,42 

- 

-.87 

1.10 

-1.32 

- 

-.76 

.26 

m 

.24 

.88 

.43 

- 

.43 

.28 

- 

-.15 

1.10 

.45 

- 

.06 

.38 

- 

-.04 

.22 

.48 

- 

.12 

-1.13 

- 

.70 

-.81 

-1.12 

- 

.78 

-.32 

- 

-.26 

-.17 

-.43 

- 

-.29 

.66 

- 

.89 

-1.06 

.47 

- 

.74 

.88 

- 

.21 

.30 

1.00 

- 

.35 

.10 

- 

-.17 

.14 

.24 

- 

.04 

.31 

- 

-.07 

.16 

.45 

- 

.15 

.27 

- 

-.51 

.23 

.38 

- 

-.29 

-.04 

m 

-.86 

.42 

.01 

- 

-.69 

-1.44 

- 

-1 . 56 

.90 

-1.52 

- 

-1.49 

*.47 

•• 

-1.47 

.63 

-.70 

mm 

-1.61 

.60 

.78 

.98 

-.36 

.44 

.72 

.74 

m 

-.12 

-.67 

.34 

- 

-.07 

-.87 

- 

-.81 

.51 

-.48 

- 

-.76 

.38 

- 

.19 

.51 

-.48 

- 

.24 

.38 

- 

-.89 

.29 

-.56 

- 

-.88 

.22 

- 

.11 

.29 

-.56 

- 

.13 

.22 

- 

-.54 

-.31 

-.37 

M 

-.58 

-.32 

- 

-.54 

-.31 

.63 

-.58 

-.32 

- 

1.54 

.49 

-.15 

- 

1.51 

.58 

- 

-.46 

.49 

-.15 

- 

-.49 

.58 

- 

- 

- 

1.12 

- 

- 

- 

- 

- 

- 

-.92 

- 

- 

- 

- 

- 

- 

-.92 

- 

- 

- 

- 

- 

- 

-1.31 

- 

- 

- 

- 

- 

- 

-2.37 

- 

- 

- 

- 

- 

- 

-.41 

- 

- 

- 

- 

- 

- 

.90 

- 

- 

- 

- 

- 

- 

.67 

- 

- 

- 

- 

- 

- 

-.31 

- 

- 

- 

- 

- 

- 

1.26 

- 

- 

- 

- 

- 

- 

1.36 

- 

- 

- 

- 

- 

- 

.24 

- 

- 

- 

- 

- 

- 

.91 

- 

- 

- 

- 

- 

- 

-.29 

- 

- 

- 

- 

- 

- 

.42 

- 

- 

- 

- 

- 

- 

.51 

- 

- 

- 
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